Unfriended friend still showing up in in-game leaderboards

Hello, I play on PC (MS Store) and recently I unfriended someone through the Xbox app. Problem is, that person is still showing up in every leaderboard and comparison (completed roads/completed boards and so on). I tried to block that person too but to no avail. What can I do ?