As many have said, i think you will notice a difference. I dind a 34t is easier than a 36t and changed by gravel bike to a 34t as a result.
You could look into buying a new crank, as mentioned. A 46/30 crank setup by FSA or Sugino. Or a 48/32 setup by Praxis. Thatll cost $200-350 depending on crank.
You could probably change the inner ring AND swap the cassette, if your drivetrain level has an 11-34 option.(10 speed or less). If its an 11speed driveteain, you could get an 11-36t cassette by SRAM like their 1170 model(i think). You may need a wolftooth roadlink, but thats easy to do and my 105 drivetrain handles an 11-36t cassette with the wolftooth. Thatd be my pick...since it is what i picked.