The idea behind cached-path is to provide a unified, simple, extendable interface for accessing both local and remote files. This can be used behind other APIs that need to access files agnostic to where they are located.
For remote files, cached-path supports several different schemes out-of-the-box in addition
s3 for AWS S3,
gs for Google Cloud Storage,
hf for HuggingFace Hub. See
cached_path.cached_path() for more details.
You can also extend cached-path to support other schemes with
cached-path is developed and maintained by the AllenNLP team, backed by the Allen Institute for Artificial Intelligence (AI2). AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering. To learn more about who specifically contributed to this codebase, see our contributors page.