For incremental updates we recommend using the from-index-date filter. The timestamp that from-index-date filters on is guaranteed to be updated every time there is a change to metadata requiring a reindex. This way you’ll pick up updated records in addition to new records.
I’m glad to hear you find the public data file useful! We’re preparing the 2022 public data file for release soon.