RSserver module diagnostics¶
Services logging used by RSserver module¶
Logging happens using logging
module.
By default, the file logs is located in the directory /var/log/aos/rs-server/
.
The directory contains the following files:
api.log
isaos-rs-server-api
service message collection file;broker-api.log
isaos-rs-broker-api
service message collection file;keystone-listener.log
isaos-rs-keystone-listener
service message collection file;project-syncer.log
is project sync message collection file;nova-listener.log
is instances sync message collection file between Nova service database and RSserver database;worker.log
is file fr collecting messages of asynchronous module tasks;trs_requests.log
is file for collecting messages about user attempts to log in and get TRS instance via TRS web interface or RSclient. Both successful and unsuccessful attempts are logged. Enabling or disabling the logging process is determined by the parameterallow_getvm_log
in the sectionbroker
in the configuration filers_server.conf
. Detailed description of theallow_getvm_log
parameter is available in the section Configuration file. The file entries have the following format:[ system time of RSserver ] - request id - connection source (web/cli) - username (which attempts to connect) - status of obtaining IP address of the TRS instance (OK/Error and also error details) - ID of groups to which the user belongs - connection project - user's IP address
profiler.log
is message collection file with API request IDs andaos-rs-worker
at each stage:- receive request;
- task creation;
- task launch;
- mutex waiting;
- mutex starting;
- sending a request to Nova to create instance;
- receiving a response (profiling the receiving process instance).
The total waiting of mutex time is also logged for each request. Messages are written to this file only in the
DEBUG
mode.
Note
Description of logging process provided by the OpenStack platform can be found in the corresponding section of the official documentation.
RSserver module debugging¶
In case of problems in operation of the module, there are following solutions:
Set log level to value
DEBUG
, which will allow logging messages about events in log files with maximum detail for diagnostics and problem solving.Run
openstack aos tests
module self-diagnosis utility.Usage example:
openstack aos tests --names rs_server --modules Modules diagnostic started. Running tests for: rs_server +--------------------------+-----------------------------------------------------------------------------------------------+ | Start date and time | Thu Dec 24 17:03:53 2020 GMT | +--------------------------+-----------------------------------------------------------------------------------------------+ | Inactive services | apache-htcacheclean, nova-novncproxy, nova-serialproxy, nova-xenvncproxy, rtslib-fb-targetctl | +--------------------------+-----------------------------------------------------------------------------------------------+ | Database names | Failed to get database names. Please run command as root. | +--------------------------+-----------------------------------------------------------------------------------------------+ | Rabbit users | Failed to get RabbitMQ info. Please run command as root. | +--------------------------+-----------------------------------------------------------------------------------------------+ | Rabbit virtual hosts | Failed to get RabbitMQ info. Please run command as root. | +--------------------------+-----------------------------------------------------------------------------------------------+ | Rabbit limits | Failed to get RabbitMQ info. Please run command as root. | +--------------------------+-----------------------------------------------------------------------------------------------+ | Apache config parameters | Failed to get Apache configs info. Please run command as root. | +--------------------------+-----------------------------------------------------------------------------------------------+ | Apache MPM | worker | +--------------------------+-----------------------------------------------------------------------------------------------+ ================ RSserver Version: 2.2.0 (latest: 2.3.0) License: 07-002-972fb12437f60c4a5411 (valid until 15.02.2021 03:00:00) ================ +--+----------------------------------------+--------+--------------------------------------------------------------+ |N |Test name | Status | Failure reason | +--+----------------------------------------+--------+--------------------------------------------------------------+ |1 |test apache config enabled |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |2 |test apache config existence |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |3 |test config file existence |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |4 |test connection to keystone |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |5 |test connection to nova |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |6 |test license validity |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |7 |test migrations applied |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |8 |test profiler log file accesses |SKIPPED |The test is relevant only if DEBUG is specified in LOG_LEVEL. | +--+----------------------------------------+--------+--------------------------------------------------------------+ |9 |test registered celery tasks |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |10|test registered tasks to schedule |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |11|test request vm log file accesses |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |12|test rs api |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |13|test rs server api connection |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |14|test rs server api service registration |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |15|test rs server systemd services |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ |16|test rs web |OK | | +--+----------------------------------------+--------+--------------------------------------------------------------+ Ran 16 in 47.537s OK (success=15, failures=0, errors=0) ====== TOTALS ====== Ran 16 in 47.537s OK (success=15, failures=0, errors=0)
Note
Detailed description of the utility is available in the section “Self-diagnosis utility”. Detailed description of the module error codes is available in the section Description of errors in calls to RSserver.