Public folders can be lightning fast or painfully slow - it's your choice. Outlook makes it optional whether you want the data in your favorite public folders downloaded and cached on your machine for fast access.
The two check boxes seen in this dialog, Download shared folders and Download Public Folder Favorites, are not checked by default. You need to open this dialog and check both of those boxes.
If you do not check the box to download public folder favorites, every time you click on the public folder, its entire contents will have to be pulled from the server, through the Internet, to your machine. That of course, is a painfully slow process.
Of course for the "download favorites" option to have any real effect, you also need to add the public folders you use to your favorites. Right click the public folder icon and choose add to favorites, or drag and drop the folder onto the favorites icon. See the illustration below.
Don't be confused by the apparent presence of two copies of each folder. The icons in the Favorites section are only shortcuts, not duplicate folders. There is only one master copy of each folder, and that is on the server. Clicking the icon in the favorites section will pull up the main and only copy of the folder. If you have selected the option to Download Public Folder Favorites, your machine will have a local copy of the folder, but the local copy is only a cache to speed up your access.