ClusterShell¶
Transport ClusterShell: worker and event handlers.
-
class
cumin.transports.clustershell.
ClusterShellWorker
(config, target)[source]¶ Bases:
cumin.transports.BaseWorker
It provides a Cumin worker for SSH using the ClusterShell library.
This transport uses the
ClusterShell
Python library to connect to the selected hosts and execute a list of commands. This transport accept the following customizations:sync
execution mode: given a list of commands, the first one will be executed on all the hosts, then, if the success ratio is reached, the second one will be executed on all hosts where the first one was successful, and so on.async
execution mode: given a list of commands, on each hosts the commands will be executed sequentially, interrupting the execution on any single host at the first command that fails. The execution on the hosts is independent between each other.custom execution mode: can be achieved creating a custom event handler class that extends the
BaseEventHandler
class defined incumin/transports/clustershell.py
, implementing its abstract methods and setting to this class object the handler to the transport.
Worker ClusterShell constructor.
- Parameters
according to parent
cumin.transports.BaseWorker.__init__()
.
-
execute
()[source]¶ Execute the commands on all the targets using the handler.
Concrete implementation of parent abstract method.
- Parameters
according to parent
cumin.transports.BaseWorker.execute()
.
-
get_results
()[source]¶ Get the results of the last task execution.
Concrete implementation of parent abstract method.
- Parameters
according to parent
cumin.transports.BaseWorker.get_results()
.
-
property
handler
¶ Concrete implementation of parent abstract getter and setter.
Accepted values for the setter: * an instance of a custom handler class derived from
BaseEventHandler
. * astr
with one of the available default handler listed inDEFAULT_HANDLERS
.The event handler is mandatory for this transport.
- Parameters
according to parent
cumin.transports.BaseWorker.handler
.
-
class
cumin.transports.clustershell.
Node
(name, commands)[source]¶ Bases:
object
Node class to represent each target node.
Additional attributes available are:
-
state
¶ cumin.transports.State
: the state of the node.
Node class constructor with default values.
- Parameters
name (str) -- the hostname of the node.
commands (list) -- a list of
cumin.transports.Command
objects to be executed on the node.
-
-
class
cumin.transports.clustershell.
BaseEventHandler
(target, commands, success_threshold=1.0, progress_bars=True, **kwargs)[source]¶ Bases:
ClusterShell.Event.EventHandler
ClusterShell event handler base class.
Inherit from
ClusterShell.Event.EventHandler
class and define a base EventHandler class to be used in Cumin. It can be subclassed to generate custom EventHandler classes while taking advantage of some common functionalities.Event handler ClusterShell extension constructor.
- Parameters
target (cumin.transports.Target) -- a Target instance.
commands (list) -- the list of Command objects that has to be executed on the nodes.
success_threshold (float, optional) -- the success threshold, a
float
between0
and1
, to consider the execution successful.progress_bars (bool) -- should progress bars be displayed
**kwargs (optional) -- additional keyword arguments that might be used by derived classes.
-
short_command_length
= 35¶ the length to which a command should be shortened in various outputs.
- Type
-
close
(task)[source]¶ Additional method called at the end of the whole execution, useful for reporting and final actions.
- Parameters
task (ClusterShell.Task.Task) -- a ClusterShell Task instance.
-
on_timeout
(task)[source]¶ Update the state of the nodes and the timeout counter.
Callback called by the
ClusterShellWorker
when aClusterShell.Task.TimeoutError
is raised. It means that the whole execution timed out.- Parameters
task (ClusterShell.Task.Task) -- a ClusterShell Task instance.
-
ev_pickup
(worker, node)[source]¶ Command execution started on a node, remove the command from the node's queue.
This callback is triggered by the ClusterShell library for each node when it starts executing a command.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_pickup()
.
-
ev_read
(worker, node, _, msg)[source]¶ Worker has data to read from a specific node. Print it if running on a single host.
This callback is triggered by ClusterShell for each node when output is available.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_read()
.
-
ev_close
(worker, timedout)[source]¶ Worker has finished or timed out.
This callback is triggered by ClusterShell when the execution has completed or timed out.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_close()
.
-
_get_log_message
(num, message, nodes=None)[source]¶ Get a pre-formatted message suitable for logging or printing.
-
_print_report_line
(message, color_func=<function ColoredType.__getattr__.<locals>.<lambda>>, nodes_string='')[source]¶ Print a tqdm-friendly colored status line with success/failure ratio and optional list of nodes.
-
_get_short_command
(command)[source]¶ Return a shortened representation of a command omitting the central part, if it's too long.
-
_commands_output_report
(buffer_iterator, command=None)[source]¶ Print the commands output in a colored and tqdm-friendly way.
- Parameters
buffer_iterator (mixed) -- any ClusterShell object that implements
iter_buffers()
likeClusterShell.Task.Task
and all the Worker objects.command (str, optional) -- the command the output is referring to.
-
_global_timeout_nodes_report
()[source]¶ Print the nodes that were caught by the global timeout in a colored and tqdm-friendly way.
-
class
cumin.transports.clustershell.
SyncEventHandler
(target, commands, success_threshold=1.0, progress_bars=True, **kwargs)[source]¶ Bases:
cumin.transports.clustershell.BaseEventHandler
Custom ClusterShell event handler class that execute commands synchronously.
The implemented logic is:
execute command #N on all nodes where command #`N-1` was successful according to batch_size.
the success ratio is checked at each command completion on every node, and will abort if not met, however nodes already scheduled for execution with ClusterShell will execute the command anyway. The use of the batch_size allow to control this aspect.
if the execution of command #N is completed and the success ratio is greater than the success threshold, re-start from the top with N=N+1.
The typical use case is to orchestrate some operation across a fleet, ensuring that each command is completed by enough nodes before proceeding with the next one.
Define a custom ClusterShell event handler to execute commands synchronously.
- Parameters
according to parent
BaseEventHandler.__init__()
.
-
start_command
(schedule=False)[source]¶ Initialize progress bars and variables for this command execution.
Executed at the start of each command.
- Parameters
schedule (bool, optional) -- whether the next command should be sent to ClusterShell for execution or not.
-
end_command
()[source]¶ Command terminated, print the result and schedule the next command if criteria are met.
Executed at the end of each command inside a lock.
-
on_timeout
(task)[source]¶ Override parent class on_timeout method to run end_command.
- Parameters
according to parent
BaseEventHandler.on_timeout()
.
-
ev_hup
(worker, node, rc)[source]¶ Command execution completed on a node.
This callback is triggered by ClusterShell for each node when it completes the execution of a command. Update the progress bars and keep track of nodes based on the success/failure of the command's execution. Schedule a timer for further decisions.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_hup()
.
-
ev_timer
(timer)[source]¶ Schedule the current command on the next node or the next command on the first batch of nodes.
This callback is triggered by ClusterShell when a scheduled Task.timer() goes off.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_timer()
.
-
class
cumin.transports.clustershell.
AsyncEventHandler
(target, commands, success_threshold=1.0, progress_bars=True, **kwargs)[source]¶ Bases:
cumin.transports.clustershell.BaseEventHandler
Custom ClusterShell event handler class that execute commands asynchronously.
The implemented logic is:
execute on all nodes independently every command in a sequence, aborting the execution on that node if any command fails.
The success ratio is checked at each node completion (either because it completed all commands or aborted earlier), however nodes already scheduled for execution with ClusterShell will execute the commands anyway. The use of the batch_size allows to control this aspect.
if the success ratio is met, schedule the execution of all commands to the next node.
The typical use case is to execute read-only commands to gather the status of a fleet without any special need of orchestration between the nodes.
Define a custom ClusterShell event handler to execute commands asynchronously between nodes.
- Parameters
according to parent
BaseEventHandler.__init__()
.
-
ev_hup
(worker, node, rc)[source]¶ Command execution completed on a node.
This callback is triggered by ClusterShell for each node when it completes the execution of a command. Enqueue the next command if the success criteria are met, track the failure otherwise. Update the progress bars accordingly.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_hup()
.
-
ev_timer
(timer)[source]¶ Schedule the current command on the next node or the next command on the first batch of nodes.
This callback is triggered by ClusterShell when a scheduled Task.timer() goes off.
- Parameters
according to parent
ClusterShell.Event.EventHandler.ev_timer()
.
-
cumin.transports.clustershell.
worker_class
¶ Required by the transport auto-loader in
cumin.transport.Transport.new()
.
-
cumin.transports.clustershell.
DEFAULT_HANDLERS
= {'async': <class 'cumin.transports.clustershell.AsyncEventHandler'>, 'sync': <class 'cumin.transports.clustershell.SyncEventHandler'>}¶ mapping of available default event handlers for
ClusterShellWorker
.- Type