The following steps describe how to create this:
Upload the template files and css
First, download the the code from the following in github: https://github.com/sregan1/SharePoint-Office365/tree/master/Sites%20Directory
Open up SharePoint Designer and select on All Files in the left hand navigation. Then go to /_catalogs/masterpage/Display Templates/Content Web Parts. Copy Control_Sites_Directory.html and Item_Sites_Directory.html here. Edit the url in these files (where it says "vivity.sharepoint.com") to your url.
Go to Site Assets, and create a Styles folder (if not there already), and copy SitesDirectory.css there.
Select the templates
First, add a content search web part to your page and edit it. In the dropdowns, select the newly added Sites Directory for the Control and Item Sites Directory for the item.
Setup the web part query
edit it, hit the Change Query button, Switch to Advanced Mode and paste in the following "contentclass:STS_Site" into the Query text box
You might want to remove some sites/paths from the query as needed by using the Property filter (e.g. -Path:https://vivity.sharepoint.com/sites/sitenottoshow). If you want to add subsites as well, make the query text (contentclass:STS_Site OR contentclass:STS_Web).
And that should be it!
NOTE: If some sites aren't showing, just kickoff a re-index of the site in Site Settings -> Search and offline availability...
The paging functionality was referenced from Sean McDonough's blog here: https://sharepointinterface.com/category/sharepoint-2007/development/