Its really hard to find dry cleaning solvents in the local stores- your better off buying online
A little dry cleaner solvent goes a LONG way. You might even just call your local place and ask if they will sell you the solvent. If you buy it online, you can end up using the rest of it as a stain remover on all clothes that are dry clean safe.
You can use lighter fluid as a substitute for the solvent. But if you are worried about the glue being effected, I wouldn't try this.
Lipsticks these days are mainly oil. I would just try the dishwashing/vinegar mixture if you are not willing to buy the solvent. Just remember to blot and end with cold water. Rubbing would make it worse and possibly stretch out your headliner.
Hope it works!