That "thing" will save you money or hard digging work one day as the drain can easily be accessed at ground level for maintenance, unblocking, cctv inspection, sampling without digging down to a regular inspection. Your the expert though...
Your vent pipe in your other image is for releasing methane gas from the drainage system and allowing fresh air in. This one is for drain access. Its more than likely installed on the sewer main drain but it 'could' be for stormwater drain. Either way, once again its fine and you should feel lucky they even put one in. I come across a lot of drains both new and old where the only access to a drain is 0.6m - 1.8 metres deep which means your digging for the rest of your afternoon or you are paying someone by the hour to do it for you.
If your withholding $5k over this then dont be surprised if they take their pipes back or take you to court. I would.
You should ask them to fill the hole in with 1.5 cubic metres of earth, spread it out nice and evenly and then pay them. if the agreement/quote was for no ground reinstatement then ask to have your flashing for your toilet pipe finished, downpipe fitted,whatever other minor details need completion then write them a cheque.