If Deku had been born with a Quirk, he would never have had the opportunity to get close to All Might. Deku would meet All Might since he is teacher a UA, but their relationship would never deepen. Deku would have control over his Quirk and would only need instruction the way other Quirk-born students require. There would be no reason for All Might to involve himself further in Deku's life or become his mentor.
Deku is bullied by several of his classmates throughout his time at school. His childhood friend Bakugo is the main bully in Deku's life. Bakugo nicknames Midoriya "Deku," meaning worthless. If Deku had manifested a Quirk along with his peers, he would not have been ostracized and bullied.
If Deku had been born with a Quirk, not only would that eliminate Bakugo's superiority complex, but the two might have developed a healthier friendship. The two could have bonded over their shared admiration of All Might, and there wouldn't be the power struggle that exists between them now.
One of Deku's most charming characteristics is his humbleness. Unlike his peers, who have a much easier time pursuing their dreams because of their Quirks, Deku has to work twice as hard. However, his dream of working as a professional hero is also made possible by the support of those around him.
However, Gran Torino, All Might's mentor, reaches out to Deku and offers to train him. Deku improves significantly under Gran Torino's tutelage because Gran Torino is familiar with One For All's power. If Deku didn't have One For All, he would have never had the chance to train with Gran Torino and learn about One For All's history.
From studying Quirks, Deku develops a very analytical mind that serves him well against his opponents. If Deku were born with a Quirk, however, it would remove his desire to want to study them, thereby eliminating one of his strongest skills.
Although we don't know much else about him, Deku's father is a fire-breather. Children usually either inherit one of their parents' Quirks, develop a combination, or display a variation of them. In Deku's case, he could have been born with the ability to breathe fire.
All For One targets Deku because of One For All. If Deku had been born with a Quirk, he would have never received One For All and perhaps would have never encountered The League of Villains. His friends would not be at risk, and he could continue his education at UA.
Midoriya's hero name would ultimately be very different had he been born with a Quirk. Bakugo wouldn't have ever given him the degrading nickname and Uraraka would have no reason to help Midoriya find a new meaning in it. Instead, Midoriya would probably pick a hero name related to his Quirk, like Yaoyorozu's "Creati".