What I have done is set up an Aurora (Serverless) MySQL Database and each EC2 instance points to this database.
ATM to reduce costs I have only one EC2 (and there is no traffic yet).
But I don't see why there can't be more instances in an autoscaling group behind a load balancer since anyway all the data is in Aurora.
Will test this out a bit more and possibly update my answer here.