Madonna, 52, is said to have guaranteed them regular contact with Mercy James, 5, after she adopted the tot from an orphanage in June 2009.
But they have not seen her since, even though Madonna has made two return visits to Malawi with the little girl.
Now they have enlisted the help of the country's leading civil rights group CILIC to prepare a legal case.
The child, whose 16-year-old mother died five days after giving birth, was raised by her grandmother and uncles, but placed in the care of the Kondanani Children's Village when they could no longer look after her.
"Mercy's family have met me several times over the past year and they have been very upset," the Daily Mail quoted Emmie Chanika, director of CILIC, as saying.
"They have a strong recollection of being told that they would be able to see Mercy and have regular contact with her - and that when she is an adult she will return to live with them in Malawi.
"I believe they have a case in law because there appears to have been a verbal contract between them and Madonna's representatives.
"I have already consulted a lawyer on their behalf. Obviously there could be problems as there is no written agreement, but the lawyer is looking into it.
"I am preparing a letter which will appeal to Madonna's lawyer Alan Chinula to intervene on the family's behalf and ask Madonna to kindly let Mercy meet her family.
"None of us desires an embarrassing and expensive court case which could humiliate or inconvenience Madonna. The best outcome would be a proper agreement to let Mercy meet her family on a regular basis," Chanika added. (ANI)