I don't find philosophical literature to be particularly useful, though formal logic is.

In particular, familiarize yourself with the most common logical fallacies, because people use them a lot. That Wikipedia page is probably not a good introduction, but you could probably learn them by selecting a few from each section each day and reading the linked articles.

To actually start applying critical thinking, go through a day, look for the things that you assume implicitly, and then ask yourself why they are so.

For example, you probably take it for granted that people drive on the right side of the road (or the left in some countries) and stop at stop signs and traffic lights, but there are countries where people feel no such obligation. Or waiting in line at the checkstand.

Note that in a lot of cases, the answer is nothing more than "because that's the way we've always done it (or, we all implicitly agree to do it)." Even if that's so, it is not a justification to stop doing it that way, unless you can prove a significant benefit of doing it the new way. Humans are social creatures, and it is a tremendous benefit when we can look at someone and immediately tell where they are fitting into a role.