Question around contractual and legal requirements

You should go through all agreements of 3rd parties included in the ISMS scope, unless some of your agreements have the same security requirements - in such a case you should review only one such agreement and use it as a representative case for all other agreements with same security requirements.

Depending upon the number of different agreements you have (the point here is not the number of agreements you have but how different they are from each other) this may be in fact a time-consuming exercise. 

In this situation, you can define some criteria to prioritize which agreements to look at first (like the ones related to the biggest 3rd parties, or those with 3rd parties with more agreements, or those related to the most important 3rd parties, etc.)