The star later learned his dad left following the tragic murder of his brother on the tough streets of Brooklyn, New York.
Jay-Z was reunited with Adnes Reeves after the release of The Black Album in 2003 - and he admits patching up his differences with his dad was vital if he was to be a good husband to wife Beyonce, who he wed in 2008.
Speaking on the Howard Stern radio show on Monday (15Nov10), he says, "I'd dropped like, seven albums. It wasn't like he saw me make a little bit of money and ran to the money, it wasn't that. He was still who he was.
"My mum actually made that (reunion) happen. She's super strong... I'd pretty much built up a wall to that emotion (father/son feelings), which wasn't healthy. I went in (to the reunion meeting) with a lot of anger, thinking I was gonna tell him off, and it didn't happen like that..."
Reeves was battling a longterm illness when he met up with Jay-Z for the first time in over two decades and the rapper was unaware he would lose his dad within weeks.
Jay-Z, 40, recalls, "He was actually a very tall, strong man, I had no idea that he would die the next month. He looked healthy and strong. He had a bad liver from drinking and doing all the other things he'd been doing his entire life. We didn't discuss it (his illness), we had so much to discuss that day."
And the superstar is thankful he found it in his heart to fully forgive Reeves.
He adds, "You don't even get to the point because you don't allow yourself. Your father, growing up, is like Superman to you, he's everything, so once you feel that sort of hurt, you never allow yourself to get close to that feeling. You don't even wanna feel that good because you know the flip-side of that. You don't let people get that close so they won't hurt you."