Systems involved
| System | Role |
|---|---|
| Jira | Source change request from the security team. |
| Freshservice | Customer-facing CR with approvals and CAB sign-off. |
| Studio inventory | The twelve target hosts, organized by customer and site. |
| Cisco IOS-XE | The actual upgrade target. |
| TFTP / SCP | Image staging path. |
Slack #fleet-upgrade-q2 | Operational channel during each window. |
| ConnectWise PSA / NetBox | CMDB updated with new firmware version per device. |
| Gmail | Pre- and post-window customer comms. |
Walkthrough
Build the upgrade plan from Jira
Copilot reads the Jira CR, lists every host tagged
cisco-edge in inventory matching the affected version, and drafts a per-customer table with current version, target version, and the right maintenance window.Generate the customer-facing CR
The Freshservice connector creates one CR per customer. Each contains the affected device, the maintenance window, the rollback path, the contact tree, and the Jira advisory link. CAB approves five at a time.
Pre-window customer email
Copilot drafts a per-customer email through Gmail 24 hours before each window: scope, expected outage, contact phone, post-window verification commitment. You review and queue.
Stage the firmware image
Copilot pushes the IOS-XE image to the local SCP server and verifies the MD5 against Cisco’s published hash. If a customer’s edge can’t reach the central SCP, it picks the local jump host instead.
Open the war room
At T-15 minutes for each window, Copilot opens a Slack thread in
#fleet-upgrade-q2, posts the device, the customer, the rollback command set, and the on-call name. Anyone joining sees the same context.Run the upgrade procedure
The
Cisco IOS-XE upgrade procedure runs against the host. Pre-checks: reachability, free flash, backup config to TFTP, save running-config. Stage commands appear in the staging panel for approval. After approval the upgrade runs, the device reloads, and the procedure waits for the OOB SSH path to come back.Post-upgrade verification
Procedure runs
show version, show ip interface brief, show bgp summary, and the customer-specific functional check. A diff of pre and post output is attached to the run.Where Studio earns its keep
- One procedure runs against twelve hosts the same way every time, so the worst window is the same as the best.
- The pre-checks are non-negotiable — Studio refuses to push the image if free flash is short or the backup didn’t complete.
- The war room thread captures the exact commands, decisions, and outputs without anyone copying terminal scrollback.
- The CMDB and the customer email both update from the same source of truth, so no one is asking which version is now running.
Related
Procedures
Build the IOS-XE upgrade procedure once and run it per host.
Shared sessions
Bring a peer into the upgrade window for two-person verification on the highest-risk devices.