My guinea pig has a dirty nose?

My guinea pig has a dirty nose every so often. He doesn't have a runny nose, but I'm wondering if it could possibly be from his hay? I buy the bales. I have been using a very damp q-tip to wipe it off. Is this a sign of something bad? We also don't use bedding we use fleece because his cage is so big.