Indexing packages that have been uploaded to packagecloud and making them available in the package manager's metadata is handled by a set of background jobs. Adding additional parallelization to this process differs depending on the package type being indexed. Certain types of package metadata cannot be handled concurrently due to technical restrictions of that metadata format.
...
In our internal tests we are able to achieve the following reindexing times for APT and YUM repositories:
- Using a c5.9xlarge AWS EC2 instance with a db.r4.xlarge RDS instance running AWS Aurora
- Reindexes an APT repository with 50,000 Debian packages in about 1min 35s.
- Reindex a YUM repository with 50,000 RPM packages in about 1min and 13s.
Debian packages and APT repositories
...
You can increase reindex speed by disabling file list metadata for your repository. This metadata is very large, takes a lot of time to generate, and is generally only used by a small percentage of users. Disabling this metadata for APT repositories will mean that users will no longer be able to run "apt file" to determine which package from the repository provided the specified file. For extremely large packages (e.g., Chef Omnibus style packages) file list metadata may not be particularly useful – a user probably knows that all file files in /opt/example are from the Example package.
...
You can increase reindex speed by disabling file list metadata for your repository. This metadata is very large, takes a lot of time to generate, and is generally only used by a small percentage of users. Disabling this metadata for YUM repositories will mean that users will no longer be able to run "yum whatprovides" to determine which package from the repository provided the specified file. For extremely large packages (e.g., Chef Omnibus style packages) file list metadata may not be particularly useful – a user probably knows that all file files in /opt/example are from the Example package.
...