Giroud's clever header in the 85th minute secured the points after Lacazette had taken just 94 seconds to make his mark with a header from Mohamed Elneny's curling cross, equalling the record for the fastest opening goal in a Premier League season.
Before that though, Vardy stunned Arsene Wenger's men with two goals and Shinji Okazaki also went on the score sheet to nearly punished Arsenal which played without star forward Alexis Sanchez because of injury. He doesn't want to leave, but at some stage I opened the door for him because I had many strikers.
Leicester will face newly promoted Brighton at the King Power Stadium in their next Premier League match.
Giroud was the match victor for Arsenal.
He said: "With Lacazette, it's simple".
With the likes of Laurent Koscielny and Shkodran Mustafi still to come back, Wenger will no doubt be hoping his side will tighten up at the back as the season progresses, while the return of Alexis Sanchez (if he stays) will obviously strengthen the Gunners' attack.
After the victory, the Arsenal boss was quick to praise his team and the impact Lacazette made on his Premier League debut.
In the 29th minute, Jamie Vardy gave the visitors a 2-1 advantage, scoring from close range after a superb ball into the area by Marc Albrighton, who exploited a blunder by Arsenal's Granit Xhaka. I feel as well that the goals we conceded, we can work together to get rid of that because one was a corner, one was a ball we lost in the build up and the third goal was on a corner again, so we conceded two goals on corners. He called Arsenal versus Leicester a microcosm of the Gunners' typical Premier League seasons: a strong start, a dip in the middle and a brilliant finish.
Craig Shakespeare's side boasted numerous players, and characteristics, that saw them shock the world to win the title in 2015/16.
Giroud was quick to hail it and told Sky Sports after the match: "Obviously I knew that I was going to have a chance so we tried to push".
'For us it was a bit disappointing with the goals that we've let in and we shouldn't really be conceding in that way, but I think we showed great determination and great togetherness to get back into the game and we won it towards the end of the game. "I don't know whether it was lack of concentration or game management".