Really this is the most likely cause.
The client is naturally CPU bound to begin with, and Voxel data isn't cheap. When you're moving quickly it's unloading and re-loading a considerable amount of voxel data, and managing entity spawning, pathing, and AI.
As a result, you have a lot of work being done unloading data from RAM, reading data from the hard drive, running calculations on the data, putting it into RAM, and sending the relative visual data to the GPU to be processed there. Most systems will have a bit of a bottleneck here, and distant terrain/POI processing adds to the workload even more.