When you face these kinds of issue always remember that you model the wall of the staircase as the staircase goes.
So the solution is when you place the ground floor wall of staircase, you cann top offset till mid landing of 1st Floor.
Then you give bottom offset on 1st floor and start it at mid landing and give top offset till 2nd floor mid landing.
Try this and place the window you will get the exact result as you want.
Regards.