My guess is that your bucket doesn't have the right permissions. Try adding this permission to your bucket (use with caution since this will make the S3 100% public):
A compatibility issue with Craft 3.4.1 and maybe earlier. Running PostgreSQL12+ (PgSQL12+) couldn't finish Craft's installation setup.
I have no idea what change in PgSQL12+ could cause this behavior but downgrading PgSQL12 to lower versions like PostgreSQL11 worked as expected.
However Craft's docs states that PostgreSQL 9.5+ should meet Craft’s server ...