I am looking for the plastic oil filter cover for Maxima 1998. See picture below. I just need the plastic cover (I don't know the exact name what is called).

I think Nissan calls it engine cover. Hit up your local junk yard and you can get one for cheap.

How important is this cover to have?

Well, it helps to prevent damage from objects bouncing around on the road.

I wouldn't worry about it too much since it's on the side, whereas the ones underneath will take most of the beating. Personally I wouldnt make a trip to the yard for just that piece, but would certainly pick one up if I was already there.

Edit sorry im looking at the pic again and seems you are missing the one underneath, pick one up.

If you've got a Nissan dealer nearby, you should be able to pick that kind of stuff up or order it for somewhere around $10-20, depending on which one it is. You get can them from places like rockauto for a good price, but the shipping costs are usually more than the price of the part.