Is it possible to not display all the different reminder lists in the infinite scroll view on the right?
Yes, in the Agenda preferences you can determine which calendars and reminder lists you’d like to be shown in the inspector. Note that this is a premium feature.