The two big ones, Rise and fall and Gathering Storm add significent changes that, IMO, greatly improve the experience. The other DLC are strictly option, though the new frontier pass adding bonus modes with silly fantastic rules such as secret societies or epic heroes can spice up a few matches and synergies.

Frankly I believe the bundles when on a good discount are the best value but if you want to just get Rise and Fall and Gathering Storm you will get a lot of content for your dollar.

I should also note that getting Gathering Storm by itself will give you Rise and Fall's mechanics included, just not the bevy of civs and wonders it contains. This is because Gathering Storm is dependant on those rules already being used.

Maybe check the Workshop, there's two Mods claiming to improve AI.

https://steamcommunity.com/sharedfiles/filedetails/?id=878717347

https://steamcommunity.com/sharedfiles/filedetails/?id=2568934685

As for the AI, the AI moves from basic threat mechanics to using the Greivances system with threat mechanics. An AI will attack you if you are militarily weak compared to their military score and they consider you in the way of their expansion (or they have a military conquest win condition in mind.

It does make them less aggressive as time passes and they start to focus on culture, science and religious victories.

Yes