My recommendation: Use PUT everywhere instead of POST. PUT has to be idempotent, so if the request fails, the client can simply post it again. This solves issue like worrying about creating a second copy of an item if a POST timed out.
Using PUT to create elements means that the client has to supply the ID, but this is easily solved by using GUIDs as IDs. Most languages have a package for create a unique GUIDs.
I don't think we should strive to remove non-idempotent cases. If something is not idempotent does not mean it is bad. It just means that request should be handled differently.
In your example (and I ask this as I remained confused after also reading SO):
Let's say that you need the client to provide the ID in the request body.
In this case, how is using PUT when creating a new resource idempotent if the ID should be unique and you have a constraint on the DB level for example?
What happens when the second call will be made with the same ID?
If I execute the following sequence:
Call 1: PUT resource + request.body {id: 1} => a new record is created so the state of the system changes
Call 2: PUT resource + request.body {id: 1} => no record is created, maybe the existing one is updated
IMO this is not idempotent nor should it be. Creating a new resource is not an idempotent operation.
I also don't like that depending on the state of the DB two requests with the same attributes will have different impacts on the system.
In my mind as a consumer of an API it is simpler: POST it creates a new record so I know what to expect, PUT updates an existing one.
Upserts are more powerful because the client can always generate a new uuid to get the POST behavior you desire, but the reverse is not the case: there is no straightforward way to safely retry timeouts, partial success, side effects, etc.
You say the same request has a different impact on the system, but what that means is the system converges to the requested state.
Maybe this is overkill if your retry strategy is to have a user in the loop, but I don't see how it's simpler for the client even without any retry behavior (if/else new/reuse uuid vs if/else put/post).
Using PUT to create elements means that the client has to supply the ID, but this is easily solved by using GUIDs as IDs. Most languages have a package for create a unique GUIDs.