In a bowl, combine the chicken, 1 tablespoon olive oil, rosemary, 1/2 teaspoon oregano, thyme, 2 tablespoons lemon juice, 3 cloves chopped garlic, 1/2 teaspoon salt, and 1/4 teaspoon black pepper. Toss everything to coat, cover and refrigerate for 1 to 6 hours.
Preheat your Traeger grill or oven to 425F.
Preheat a large oven-safe pan over medium heat. Add the remaining 2 tablespoons of olive oil and heat until shimmering. Gently shake off any excess marinade from the chicken and carefully add it to pan, skin side-down first. Cook, undisturbed, until the skin is golden brown, 6 to 8 minutes. Transfer the chicken to a plate, skin side-up, cover and set aside.
Return the pan to a medium heat and add the chicken broth. Use a wooden spoon or spatula to lift any brown bits off the bottom of the pan. Add the potatoes, remaining 3 cloves chopped garlic, mustard, 2 tablespoons lemon juice and 1/2 teaspoon oregano. Transfer the potatoes to the Traeger grill or oven and cook until the potatoes are golden and just barely fork-tender, around 40 minutes.
Nestle the chicken into the potatoes so they’re partially submerged in the broth. Scatter the olives overtop and return the pan to the grill or oven. Cook until the potatoes are completely fork-tender and the thickest part of the chicken registers 165F, around 12 more minutes.
Remove the pan from the oven, garnish with chopped parsley and serve with the Garlicky Yogurt Sauce.