...and when the windows are closed, air is getting sucked through the stove downstairs and moves up through the house, heading for those upstairs windows.
You can attack this in several ways, but I don't think the chimney is a huge problem.
1) Stop / reduce any chances for air to exit on the upper levels...seal windows, doors, attic access, attic fans, air returns leading to the attic. This will help reduce the back-pull on the stove.
2) Break any remaining 'stack effect' in the house if possible. Close doors you don't need open - especially ones leading up / out of the stove room. Though if you have an option to open doors on a lower level, that may help too...you might get air flow coming in from other rooms to dilute the stove heat and reduce the stack effect.
3) Mix the air better in the stove room. If the entire room is ~70ºF, then that air is less likely to rise out of the room and create an internal 'stack effect' than if you had 100ºF air at the ceiling and 65ºF air at the floor. Even a ceiling fan on low may be enough to break the 'hot air rising' cycle.
4) "Stuff" the stove room with positive pressure. Even a small fan setting outside the stove room door could create a bit of positive pressure and stop the back flow of the flue pipe. If you can aim the fan at the top of the door, that would be even better as that is where the hot air would typically like to exit. (aiming at the bottom or floor, may make things a bit worse.
Good luck and stay safe!