There is an in-memory cache which defaults to 10-minutes, I don’t know offhand whether this is 10-minutes since the first read, or last access. I’m not aware of a persistent cache of LDAP users.
The UI when looking at LDAP users is querying from the remote and should not be affected by the in-memory cache.