The Scheduler Module

This module is used to deal with anything related to test scheduling/partitioning in stestr.

stestr.scheduler.generate_worker_partitions(ids, worker_path, repository=None, group_callback=None, randomize=False)[source]

Parse a worker yaml file and generate test groups

Parameters:
  • ids (list) – A list of test ids too be partitioned
  • worker_path (path) – The path to a worker file
  • repository – A repository object that will be used for looking up timing data. This is optional, and also will only be used for scheduling if there is a count field on a worker.
  • group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker. This is optional and also will only be used for scheduling if there is a count field on a worker.
  • randomize (bool) – If true each partition’s test order will be randomized. This is optional and also will only be used for scheduling if there is a count field on a worker.
Returns:

A list where each element is a distinct subset of test_ids.

stestr.scheduler.local_concurrency()[source]

Get the number of available CPUs on the system.

Returns:An int for the number of cpus. Or None if it couldn’t be found
stestr.scheduler.partition_tests(test_ids, concurrency, repository, group_callback, randomize=False)[source]

Partition test_ids by concurrency.

Test durations from the repository are used to get partitions which have roughly the same expected runtime. New tests - those with no recorded duration - are allocated in round-robin fashion to the partitions created using test durations.

Parameters:
  • test_ids (list) – The list of test_ids to be partitioned
  • concurrency (int) – The concurrency that will be used for running the tests. This is the number of partitions that test_ids will be split into.
  • repository – A repository object that
  • group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker.
  • randomize (bool) – If true each partition’s test order will be randomized
Returns:

A list where each element is a distinct subset of test_ids, and the union of all the elements is equal to set(test_ids).