That is exactly why this issue baffles me. Poor writing, character buildup maybe, but how can Arthur trust Merlin to take him away with him all the time, in missions where servant is really not needed and listen to him once and the flop over and call him useless idiot other times. I doubt trained warriors like him and his elite knights would make good tasty food priorities on missions far more important.
If Arthur was going on a visit to a neighbouring kingdom or something, and expected to be away for a period of at least several days, it would make sense for him to want to bring Merlin with him to attend him. However, it would be a different story when it comes to patrols, fighting monsters, etc, because he had no reason to believe that Merlin would be anything but a liability in a combat situation. Even going on hunts would be a stretch. Yes, there were usually people coming along as beaters and to carry back any dead animals but you wouldn't expect the Prince/King's manservant to be part of that because he simply wouldn't have the time.
Look at Merlin's duties as Arthur's servant; he cleaned his chambers, prepared and served his meals, did his laundry, prepared his baths, cared for and walked his dogs, was responsible for his horses and stables, and was responsible for maintaining his armour and weapons. He also acted as his squire during training sessions and tournaments, and was loaned out to visitors in need of a servant. In addition, he acted as Gaius' assistant, gathering medicinal plants on a regular basis, doing chores in their quarters and cleaning out the leech tank. Riding out with Arthur would easily take up most of a day, making it impossible for Merlin to complete half of his chores, so it would make no sense for Arthur to drag his servant away from his duties when he clearly did not consider him to be an asset on such trips. He even said that he didn't know why he brought Merlin along.
Had Uther appointed Merlin to be Arthur's squire, it would make sense for him to be dragged along whenever Arthur was out and about. If Arthur found him more a hindrance than a help, it could be handwaved because the job was given to Merlin as a reward for saving Arthur's life, so he was allowed to keep it even if he was deemed useless enough to be fired under other circumstances. However, his job as Arthur's manservant should have been an indoor one.
FImber btw, loads of young kindgs actually took part in some adventures. I guess its coz of hot blood and youth.
That, and a King couldn't allow himself to be seen as a coward leaving other men to do all of the fighting, not in his youth, at any rate. It's why Uther often sent Arthur on missions, despite the risk to his life. He could have kept him safe inside the castle - or at least tried to - but that would make Arthur look weak, and people wouldn't want to follow a weak King. They just needed a balance between having Arthur risk his life when the Prince/King needed to be seen to take an active role in the defence of the kingdom, and avoiding unnecessary risk. Uther forbidding Arthur to save Merlin in The Poisoned Chalice is an example of this stance.