Attila and Iidiko Wurth could be kicked out of their Hammersmith home with their 15-month-old daughter and three-year-old son.
They say they received notice from their management firm that if the noise continued they would be given two weeks notice to leave.
Talking to BBC, the young couple said it was a 'horrible discrimination' against families with children who rent.
The management firm said neighbours had complained of noisy children in the early hours of the morning, adding in an email to the couple: 'We have subsequently liaised with your landlord and are instructed that we are to agree arrangements with you to vacate the property as soon as possible.'
The family have been told to keep movement and noise to a minimum but remain frightened about their future in the block.
He is adamant they have been careful about the noise they make and do not have a stereo or TV, but added it is unrealistic to think a baby won't cry.
The BBC reported the managing agent, Sheraton Management Ltd, said the Wurths 'were in breach of contract as they were causing disturbance to the other occupants of the building... not only relating to noisy children, but also other noise nuisance'.