Ferguson signed de Gea in 2011 and he had big shoes to fill following the retirement of Edwin van der Sar.
The Spain international struggled to adapt to the physicality of the Premier League in his first few years and even had spells out of the team.
He quickly established himself as one of the world's best goalkeepers however and Neville, who had been critical of de Gea in the past, was given both barrels by his former manager.
Neville, speaking on the Overlap in partnership with Sky Bet explained: "Sir Alex rang me once on my comments on David De Gea. Once.
"It was in the second season of my punditry, so he let the first season go.
"Do you remember, his first season was a massive struggle, wasn't it? But in the second season, he started to improve, to the point he became one of the best goalkeepers in the world.
"Sir Alex rang me and said 'I got it last year, but I think you're being f***ing unfair son'."
Ferguson is the only manager Neville played under at club level in his 19 year career and has an unlimited amount of respect for his former boss.
He believes it is important for pundits to express their opinion, but took Ferguson's word on board.
De Gea has underperformed at times this season and came under heavy criticism following a 4-0 defeat to Brentford earlier this month.
The former Atletico Madrid man isn't the best with the ball at his feet and played United into trouble for the Bees' second goal, after letting Josh Dasilva's opener slip from his grasp eight minutes prior.
United manager Erik ten Hag has backed his goalkeeper however and believes he has all the tools to play his style of football.
"I am convinced he can do it," ten Hag maintained. "And it's obvious the second goal, okay (he was at fault). I have already seen in the first games and training he can do that."
United registered their first win of the season against Liverpool on Monday and return to action on Saturday against Southampton.
New signing Casemiro could be in line for his first league start, but it's likely ten Hag sticks with the team that defeated Liverpool five days ago.
Featured Image Credit: Alamy & The Overlap
- Financial expert identifies worrying change at Man Utd since Sir Alex Ferguson's retirement
- Man Utd broke Sir Alex Ferguson’s golden rule when they gave Erik ten Hag his contract
- Rare footage of Paul Pogba breaking down Sir Alex Ferguson's management style will get people talking
- Sir Alex Ferguson's epic reaction to Eric Cantona's 'kung-fu kick' sums up his managerial style