The Scheduler Module¶
This module is used to deal with anything related to test scheduling/partitioning in stestr.
-
stestr.scheduler.
generate_worker_partitions
(ids, worker_path, repository=None, group_callback=None, randomize=False)[source]¶ Parse a worker yaml file and generate test groups
Parameters: - ids (list) – A list of test ids too be partitioned
- worker_path (path) – The path to a worker file
- repository – A repository object that will be used for looking up timing data. This is optional, and also will only be used for scheduling if there is a count field on a worker.
- group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker. This is optional and also will only be used for scheduling if there is a count field on a worker.
- randomize (bool) – If true each partition’s test order will be randomized. This is optional and also will only be used for scheduling if there is a count field on a worker.
Returns: A list where each element is a distinct subset of test_ids.
-
stestr.scheduler.
local_concurrency
()[source]¶ Get the number of available CPUs on the system.
Returns: An int for the number of cpus. Or None if it couldn’t be found
-
stestr.scheduler.
partition_tests
(test_ids, concurrency, repository, group_callback, randomize=False)[source]¶ Partition test_ids by concurrency.
Test durations from the repository are used to get partitions which have roughly the same expected runtime. New tests - those with no recorded duration - are allocated in round-robin fashion to the partitions created using test durations.
Parameters: - test_ids (list) – The list of test_ids to be partitioned
- concurrency (int) – The concurrency that will be used for running the tests. This is the number of partitions that test_ids will be split into.
- repository – A repository object that
- group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker.
- randomize (bool) – If true each partition’s test order will be randomized
Returns: A list where each element is a distinct subset of test_ids, and the union of all the elements is equal to set(test_ids).