The problem must be with the choke / carb
It runs bad regradless the airfilter is on or removed.
Only with the complete airbox removed it runs ok.
I have reinforced the tube from the airfilterbox to carb to rule out that it possibel is sucked flat.
I can barely hear a little change in rpm when flipping on the choke

 

sounds like ur chicken is choked

 

Make sure you have a little play at the choke lever since you replace the plunger.If not the choke could be partially on,but since you say no change in the engine when on,problem is likely elsewhere. On something like this,setting for quite awhile I'd still soak the carb in Berrymans or take it and have it vatted. Sometimes it's the only good way of getting all the passages clean,then blow them out with compressed air and retrace your steps on the carb rebuild. I know you said you rebuilt it,but Shindy carb kits are the best. Worst case scenario there are cheap knock off carbs on ebay if you get tired of fighting it. New Polaris Sportsman 500 Carburetor 4x4 ATV Quad Carb 1996 1998 Non HO Y | eBay

 

Cracked the code
Put the whole airfilterbox together and it runs fine.
Removed the foam filter and it bogs, when restricting airflow simulating foam filter on with a finger its all ok.

 

Polaris jetted the carbs with the air boxes,filters and foam prefilter to be used or they can run lean and bog. Never did like this prefilter on the intake tube as the seal support could creep up and could block air flow and make them run like crap also. Dirt Cheap Yamaha, Polaris, Suzuki & Kawasaki OEM Parts & Accessories – Cycle Parts Warehouse They finally started doing things right starting with the 99 Magnum 500 by just having a prefilter around the air filter itself same as the Sportsman.Dirt Cheap Yamaha, Polaris, Suzuki & Kawasaki OEM Parts & Accessories – Cycle Parts Warehouse Every year seems like they kept experimenting with air box designs,filters and intake tubes. Glad you found it was a simple fix.