check-restart

check-restart

Go-based tooling used to detect whether a restart (service) or reboot (system) is needed.

Table of Contents

• Project home
• Overview
• Features
• Changelog
• Requirements
  • Building source code
  • Running
• Installation
  • From source
  • Using release binaries
  • Deployment
• Configuration
  • Command-line arguments
    • check_reboot
  • Logging output
• Examples
  • OK result
    • Nothing found
    • Problematic assertion listed but evaluation result ignored
    • Disable default ignored assertion path entries
  • WARNING result
    • Without verbose flag
    • Verbose output
  • CRITICAL result
• License
• References

Project home

See our GitHub repo for the latest code, to file an issue or submit improvements for review and potential inclusion into the project.

Overview

This repo is intended to provide various tools used to detect whether a restart (service) or reboot (system) is needed.

Features

• Nagios plugin (check_reboot) for monitoring "reboot needed" status of Windows systems

  • NOTE: The intent is to support multiple operating systems, but as of this writing Windows is the only supported OS
    • see https://github.com/atc0005/check-restart/labels/linux

• Optionally list ignored assertions

  • ignored assertions are not shown by default

• Optionally disable list of default ignored paths

  • by default a small list of ignored paths are used to prevent known problematic assertion matches from affecting service check results

• Optional branding "signature"

  • used to indicate what Nagios plugin (and what version) is responsible for the service check result

• Optional, leveled logging using rs/zerolog package

  • logfmt format output (to stderr)
  • choice of disabled, panic, fatal, error, warn, info (the default), debug or trace.

Changelog

See the CHANGELOG.md file for the changes associated with each release of this application. Changes that have been merged to master, but not yet an official release may also be noted in the file under the Unreleased section. A helpful link to the Git commit history since the last official release is also provided for further review.

Requirements

The following is a loose guideline. Other combinations of Go and operating systems for building and running tools from this repo may work, but have not been tested.

Building source code

• Go
  • see this project's go.mod file for preferred version
  • this project tests against officially supported Go releases
    • the most recent stable release (aka, "stable")
    • the prior, but still supported release (aka, "oldstable")
• GCC
  • if building with custom options (as the provided Makefile does)
• make
  • if using the provided Makefile

Running

• Windows 8.1

• Windows 10

• Windows Server 2012 R2

• Windows Server 2016

• Windows Server 2019

• Windows Server 2022

Installation

From source

1. Download Go
2. Install Go
3. Clone the repo
   1. cd /tmp
   2. git clone https://github.com/atc0005/check-restart
   3. cd check-restart
4. Install dependencies (optional)
   • for Ubuntu Linux
     • sudo apt-get install make gcc
   • for CentOS Linux
     1. sudo yum install make gcc
5. Build
   • manually, explicitly specifying target OS and architecture
     • GOOS=windows GOARCH=amd64 go build -mod=vendor ./cmd/check_reboot/
       • most likely this is what you want
       • substitute amd64 with the appropriate architecture if using different hardware (e.g., arm64)
   • using Makefile windows recipe
     • make windows
       • generates x86 and x64 binaries
   • using Makefile release-build recipe
     • make release-build
       • generates the same release assets as provided by this project's releases
6. Locate generated binaries
   • if using Makefile
     • look in /tmp/check-restart/release_assets/check_reboot/
   • if using go build
     • look in /tmp/check-restart/
7. Copy the applicable binaries to whatever systems needs to run them so that they can be deployed

NOTE: Depending on which Makefile recipe you use the generated binary may be compressed and have an xz extension. If so, you should decompress the binary first before deploying it (e.g., xz -d check_reboot-windows-amd64.exe.xz).

Using release binaries

1. Download the latest release binaries
2. Decompress binaries
   • e.g., xz -d check_reboot-windows-amd64.exe.xz
3. Copy the applicable binaries to whatever systems needs to run them so that they can be deployed

Deployment

1. Place check_reboot in a location where it can be executed by the monitoring agent

   • Usually the same place as other Nagios plugins
   • For example, on a default Windows system with NSClient++ x64 installed the check_reboot.exe plugin would be deployed to C:\Program Files\NSClient++\scripts\custom\check_reboot.exe

2. Update the monitoring agent configuration configuration (on a system that you wish to monitor for "needs reboot" condition) to create a new command definition

   • We will pretend that the DNS name for the system is windows.example.com

   • [/settings/external scripts/scripts]
     ; NOTE: stderr output is returned mixed in with stdout content. Disable logging to prevent this.
     ; check_reboot=scripts\\custom\\check_reboot.exe --verbose --show-ignored --log-level disabled
     ;
     ; By default specific paths are ignored. You can optionally disable the default ignored paths.
     ; check_reboot=scripts\\custom\\check_reboot.exe --verbose --show-ignored --disable-default-ignored
     ;
     check_reboot=scripts\\custom\\check_reboot.exe --verbose --show-ignored
     

   • see NSClient++ External scripts doc for additional details

3. Restart the nscp service (label of NSClient++ (x64))

4. Create a new Nagios "console" command definition to allow requesting the monitoring agent to run the plugin

   • define command {
         # $ARG1$ is an optional non-default port
         command_name    wcheck_reboot
         command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -p $ARG1$ -t 30 -c check_reboot
     }
     

5. Create a new Nagios "console" service check that requests the monitoring agent to execute the plugin

   • define service {
         host_name              windows.example.com
         service_description    check reboot
         use                    generic-service
         check_command          wcheck_reboot!12345!!!!!!!
         notification_period    24x7withMaintenanceWindow
         contacts               atc0005
         register               1
     }
     

Configuration

Command-line arguments

• Use the -h or --help flag to display current usage information.
• Flags marked as required must be set via CLI flag.
• Flags not marked as required are for settings where a useful default is already defined, but may be overridden if desired.

check_reboot

Logging output

Early testing using NSClient++ suggests that both stderr and stdout are mixed together and sent back to the Nagios console as one output stream.

If this becomes a problem you can disable logging output (--log-level disabled) to prevent stderr from being including in the output sent back to the Nagios console.

If you know of a way to prevent stderr from being mixed into the output sent back to Nagios, please file a GitHub issue in this project to share your findings.

Examples

OK result

Nothing found

No reboot needed.

This output is emitted by the plugin when a reboot is not needed.

C:\Users\Administrator>"C:\Program Files\NSClient++\scripts\custom\check_reboot.exe"
OK: Reboot not needed (assertions: 15 applied, 0 matched, 0 ignored)

Reboot not required

 | 'errors'=0;;;; 'evaluated_assertions'=15;;;; 'evaluated_file_assertions'=1;;;; 'evaluated_registry_assertions'=14;;;; 'ignored_assertions'=0;;;; 'matched_assertions'=0;;;; 'time'=42ms;;;;

Regarding the output:

• The last line beginning with a space and the | symbol are performance data metrics emitted by the plugin. Depending on your monitoring system, these metrics may be collected and exposed as graphs/charts.
• This output was captured on a Windows 10 system, but is comparable to the output emitted by other Windows desktop & server systems.

Problematic assertion listed but evaluation result ignored

C:\Users\Administrator>"C:\Program Files\NSClient++\scripts\custom\check_reboot.exe" --verbose --show-ignored
OK: Reboot not needed (assertions: 15 applied, 0 matched, 1 ignored)

Reboot not required

Assertions ignored:

  - Subkeys for key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/WindowsUpdate/Services/Pending found
    subkey: 117cab2d-82b1-4b5a-a08c-4d62dbee7782


 | 'errors'=0;;;; 'evaluated_assertions'=15;;;; 'evaluated_file_assertions'=1;;;; 'evaluated_registry_assertions'=14;;;; 'ignored_assertions'=1;;;; 'matched_assertions'=0;;;; 'time'=1ms;;;;

Regarding the output:

• The last line beginning with a space and the | symbol are performance data metrics emitted by the plugin. Depending on your monitoring system, these metrics may be collected and exposed as graphs/charts.
• The --verbose flag is used to display additional details (where available) for an assertion match.
• The --show-ignored flag is used to display assertions whose evaluation results were recorded, but are ignored when determining final plugin state.
• This output was captured on a Windows Server 2012 system, but is comparable to the output emitted by other Windows desktop & server systems.

Disable default ignored assertion path entries

This example uses the --disable-default-ignored flag to skip using the default ignored assertion path entries. This can be useful for cases where a sysadmin wants to apply the assertion match results for all discovered paths, even ones historically found to be false-positives or otherwise unreliable.

C:\Users\Administrator>"C:\Program Files\NSClient++\scripts\custom\check_reboot.exe" --verbose --disable-default-ignored
WARNING: Reboot needed (assertions: 15 applied, 1 matched, 0 ignored)

**ERRORS**

* reboot assertions matched, reboot needed

**DETAILED INFO**

Reboot required because:

  - Subkeys for key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/WindowsUpdate/Services/Pending found
    subkey: 117cab2d-82b1-4b5a-a08c-4d62dbee7782


 | 'errors'=0;;;; 'evaluated_assertions'=15;;;; 'evaluated_file_assertions'=1;;;; 'evaluated_registry_assertions'=14;;;; 'ignored_assertions'=0;;;; 'matched_assertions'=1;;;; 'time'=0ms;;;;

Regarding the output:

• The last line beginning with a space and the | symbol are performance data metrics emitted by the plugin. Depending on your monitoring system, these metrics may be collected and exposed as graphs/charts.
• The --verbose flag is used to display additional details (where available) for an assertion match.
• The --disable-default-ignored flag indicates that the default ignored path entries will not be used; all discovered paths will be used to determine final plugin state.
• This output was captured on a Windows Server 2012 system, but is comparable to the output emitted by other Windows desktop & server systems.

WARNING result

This output is emitted by the plugin when a reboot is needed.

Without verbose flag

C:\Users\Administrator>"C:\Program Files\NSClient++\scripts\custom\check_reboot.exe"
WARNING: Reboot needed (assertions: 15 applied, 5 matched, 0 ignored)

**ERRORS**

* reboot assertions matched, reboot needed

**DETAILED INFO**

Reboot required because:

  - Value PendingFileRenameOperations of type MULTI_SZ for key HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/Session Manager found

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/WindowsUpdate/Auto Update/RebootRequired found

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Component Based Servicing/RebootPending found

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Component Based Servicing/PackagesPending found

  - File C:/Windows/WinSxS/pending.xml found


 | 'errors'=0;;;; 'evaluated_assertions'=15;;;; 'evaluated_file_assertions'=1;;;; 'evaluated_registry_assertions'=14;;;; 'ignored_assertions'=0;;;; 'matched_assertions'=5;;;; 'time'=1ms;;;;

Regarding the output:

• The last line beginning with a space and the | symbol are performance data metrics emitted by the plugin. Depending on your monitoring system, these metrics may be collected and exposed as graphs/charts.
• This output was captured on a Windows Server 2022 system, but is comparable to the output emitted by other Windows desktop & server systems.

Verbose output

C:\Users\Administrator>"C:\Program Files\NSClient++\scripts\custom\check_reboot.exe" --verbose
WARNING: Reboot needed (assertions: 15 applied, 5 matched, 0 ignored)

**ERRORS**

* reboot assertions matched, reboot needed

**DETAILED INFO**

Reboot required because:

  - Value PendingFileRenameOperations of type MULTI_SZ for key HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/Session Manager found
    Entries [14 total, 12 skipped]: C:/Program Files (x86)/Microsoft/EdgeUpdate/1.3.167.21, C:/Windows/Temp/f7cbd550-567a-4a88-a3e4-c67b414b439d.tmp

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/WindowsUpdate/Auto Update/RebootRequired found

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Component Based Servicing/RebootPending found

  - Key HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Component Based Servicing/PackagesPending found

  - File C:/Windows/WinSxS/pending.xml found


 | 'errors'=0;;;; 'evaluated_assertions'=15;;;; 'evaluated_file_assertions'=1;;;; 'evaluated_registry_assertions'=14;;;; 'ignored_assertions'=0;;;; 'matched_assertions'=5;;;; 'time'=0ms;;;;

Regarding the output:

• The last line beginning with a space and the | symbol are performance data metrics emitted by the plugin. Depending on your monitoring system, these metrics may be collected and exposed as graphs/charts.
• The --verbose flag is used to display additional details (where available) for an assertion match.
• This output was captured on a Windows Server 2022 system, but is comparable to the output emitted by other Windows desktop & server systems.

CRITICAL result

This result is returned when an error occurs during the attempt to determine whether a reboot is needed.

TODO: Provide example output when this scenario is encountered.

License

See the LICENSE file for details.

References

• https://adamtheautomator.com/pending-reboot-registry/
• https://github.com/adbertram/Random-PowerShell-Work/blob/master/Random%20Stuff/Test-PendingReboot.ps1