Copyright and Your Dissertation or Thesis: Ownership, Fair Use, and Your Rights and Responsibilities by Kenneth Crews is a must read resource for graduate students as they prepare their thesis or dissertation. In it, you will find information about fair use as it specifically applies to theses and dissertations, including a more detailed breakdown of the four factors and examples of fair uses.
As you consider your uses of others' works, you may also find it helpful to refer to various statements of best practices that describe situations considered to be fair use of copyrighted works by various scholarly communities. While a statement of best practices might not exist for your specific discipline, these statements may help guide your thinking, as there are many scholarly uses that are considered fair across disciplines.
You might also find these tools helpful when you are analyzing your proposed use: