This is one of the things that bothered me the most from existing React libraries, if you wanted to update a single query parameter now you needed to do a lot of extra work. It bothered me so much I ended up making a library around this [1], where you can do just:
[2] https://codesandbox.io/p/sandbox/festive-murdock-1ctv6