The man, originally from Eritrea, was charged with rape and placed into custody while police investigate the case.
A witness heard the woman crying out for help and called the police.
The victim, who lives in a nursing home, had to receive medical treatment following the attack and was released from hospital on Tuesday.
The abused pensioner was visiting the cemetery to visit her sister’s grave in the town of Iddenbueren when the reported attack happened.
The alleged rapist arrived in Germany in 2013 and was living just 15 minutes away from the scene of the attack in the nearby refugee shelter in Horstel.
If found guilty, he could face a total of 15 years behind bars.