Fill a medium sauce pan with water and bring to a boil over high heat. Season the water with 2 teaspoons salt and add the orzo. Cook according to package instructions or until the orzo is barely al dente. Strain, rinse with water to prevent clumping, and set aside.
Add olive oil to a saute pan and heat over medium heat until shimmering. Add the carrot and onion and season with a pinch of salt. Cook, stirring, until slightly softened, around 6 minutes. Add the mushrooms and cook, stirring until slightly softened, around 5 minutes. Add the garlic and cook, stirring, 60 seconds.
Add the chicken stock and kale. Raise the heat to high and bring to a simmer. Cook until the carrots are soft, around 8 minutes. Add the chicken, lemon zest and juice.
Add the reserved orzo and stir through to combine. Taste for seasoning and adjust with salt and pepper as desired. Ladle the soup into individual serving bowls and garnish with fresh dill.