If you want to get picky, yes dirt can pose a potential long term problem; however I wouldn't loose sleep over it. That said, my personal issue with the K&N, [or *any* oiled filter, or filter that replaces the stock air box for that matter] is two fold. First is, by removing the stock air box, you'll likely loose power rather than gain any. This is due to the hotter air under the hood. The stock air-box channels cold air from outside the car. When you remove the air box, you're no longer getting cold, outside air, which is more dense and higher in oxygen.
My second issue is with the MAF. [Mass Air Flow meter] The car's ECU [computer] uses the MAF to determine air/fuel mixture. Most MAFs use a small heated wire to determine the volume of air. When air flows over the wire, will effect how much current must pass through it to maintain a certain temp. Switching to an oiled filter wil often lead to an oily build up on the wire, which at the very least, can impede performance and at the worst, ruin your MAF. A new MAF can cost more than $1000 to replace and is not covered under warranty. The best way to prevent this is *NOT* over oil the filter after you clean it *AND* get a can of
MAF Cleaner from your local auto supply store and be sure to clean the MAF after every filter cleaning. ***WARNING*** DO NOT USE ANYTHING OTHER THAN MAF CLEANER! i. e. Such as Brake Clean or Carb Clean. Most solvents can and will damage the MAF. MAF Cleaner is designed specifically NOT to harm the MAF.