Scheduler module diagnostics

Important

All commands are executed only from superuser.

Mode superuser:

sudo -i

Services logging used by Scheduler module

Logging happens using logging Scheduler module.

The logging level is defined in the configuration file.

Logging is the recording of processes and errors during the operation of the module in files with the extension .log. The following files exist:

  • aos-scheduler-beat is responsible for creating and modifying tasks, as well as adding tasks to the queue. All information from these processes gets in log file:
    • /var/log/aos/scheduler/beat.log.
  • celery-worker is responsible for for the process of executing the task that is in the queue. Logging task progress:
    • /var/log/aos/scheduler/worker.log.
  • Process: term: REST API runs as systemd service:
    • Log of requests is:
      • /var/log/aos/scheduler/api.log.

Note

Description of logging process provided by the OpenStack platform can be found in the corresponding section of the official documentation.

Scheduler module debugging

In case of problems in operation of the module, there are following solutions:

  1. Set log level to value DEBUG, which will allow logging messages about events in log files with maximum detail for diagnostics and problem solving.

  2. Run openstack aos tests module self-diagnosis utility.

    Usage example:

    openstack aos tests --names scheduler --modules
    Modules diagnostic started.
    Running tests for: scheduler
    
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Start date and time      | Thu Dec 24 17:03:53 2020 GMT                                                                  |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Inactive services        | apache-htcacheclean, nova-novncproxy, nova-serialproxy, nova-xenvncproxy, rtslib-fb-targetctl |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Database names           | Failed to get database names. Please run command as root.                                     |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Rabbit users             | Failed to get RabbitMQ info. Please run command as root.                                      |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Rabbit virtual hosts     | Failed to get RabbitMQ info. Please run command as root.                                      |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Rabbit limits            | Failed to get RabbitMQ info. Please run command as root.                                      |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Apache config parameters | Failed to get Apache configs info. Please run command as root.                                |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    | Apache MPM               | worker                                                                                        |
    +--------------------------+-----------------------------------------------------------------------------------------------+
    
    ================
    Scheduler
    Version: 2.2.0 (latest: 2.3.0)
    License: This module does not require a license.
    ================
    +---+-----------------------------------------+--------+-----------------+
    | N | Test name                               | Status | Failure reason  |
    +---+-----------------------------------------+--------+-----------------+
    | 1 | test config file existence              | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 2 | test connection to keystone             | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 3 | test connection to rabbit               | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 4 | test migrations applied                 | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 5 | test scheduler api connection           | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 6 | test scheduler api service registration | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    | 7 | test scheduler systemd services         | OK     |                 |
    +---+-----------------------------------------+--------+-----------------+
    Ran 7 in 18.857s
    OK (success=7, failures=0, errors=0))
    
    ======
    TOTALS
    ======
    Ran 7 in 18.857s
    OK (success=7, failures=0, errors=0))
    

Note

Detailed description of utility is available in section «Self-diagnosis utility».