I have found a solution to this problem for myself that I thought I'd share. If it helps any of you guys out, that would be great.
In short, I have found that this issue seems related to the in-game V-Sync.
The solution, for me, is as follows;
-Disable Dynamic Optimisation and set a Custom graphics profile. I am running with everything maxed as far as it will go @ 4K. (Depending on your hardware, you will have to find suitable settings. Look at Hardware Unboxed's YouTube videos on optimisation and GPU benchmarks for Forza Horizon 4. Very, very helpful!)
-Disable V-Sync in game.
-Set FPS cap in game,
-If you are having issues with screen tearing and you need to use V-Sync, use the NVIDIA control panel V-Sync. I am using the Fast setting.
This has completely eliminated the streaming errors for me. Everything is as smooth as butter, and I don't need to change priorities in task manager at all.
I am playing at 4K with everything cranked to the max, and have no issues at all in the areas of the map where this error would surface.
Also, I have noticed that running multiple displays, (I have a 1440p monitor as secondary), confuses the hell out of this app. To get everything rock solid stable, I just run the one monitor, with next to no background applications.
Hope this helps someone, because I was absolutely infuriated trying to get this to run properly.
i7 6700K
16GB RAM
ASUS MAXIMUS VIII FORMULA
Strix GTX 1080 Ti
850 EVO 500GB