How to escape double quotes in ARM templates

This will be a very short post, but it’s something that drove me crazy while adding OMS deployment and configuration to my automated deployment and configuration of our internal application and underlying infrastructure using PowerShell, PowerShell DSC and ARM templates.  As a part of this deployment, I wanted to include saved queries which are required to create alerts, but I was unable to figure out how to escape the double quotes needed in OMS search for queries such as the following:

As you can see, JSON does not like double quotes within double quotes, which makes sense.  The question is how do we escape the double quotes?  After a bit of research I was able to track down a sample template and find the answer.  Simply add a back-slash before each double quote and the issue is solved!  See below for an example.

Hopefully this will save somebody a bit of time in the future as we inevitably move towards increased automation!

 

Monitor and Recover Stopped Automatic Services with OMS – Part 1

I was working with a customer recently and one of the asks was to configure OMS to monitor for stopped automatic services on servers throughout the environment.  My first thought was that we could easily use the data collected by the Configuration Tracking solution and configure queries to alert when a service is stopped.  Unfortunately, although Configuration Tracking is a great solution, for this purpose it did not meet the requirements due to the 1 hour data collection interval.  We needed to be notified of the critical service stopping as close to real-time as possible.  Plan B was to utilize Event ID 7024 and custom fields as we were already collecting the Application log.  However, during my testing on Windows Server 2012 R2, the only event logged to the Application log when a service was stopped manually was Event ID 1.  Further, what if a service just doesn’t start after reboot?   Once again, there may be no events logged as technically there could be no error. 

SO…although technically both of the other options could work in certain scenarios, in this particular case we needed something a bit more granular.  Time for some fun with PowerShell, Azure Automation and the Data Collector API!

Schedule an Azure Automation Runbook Using Minutes

I was working with a customer recently and we realized that when using the “Schedule” functionality the most granular re-occurrence interval available is 1 hour.  In this particular case, we needed to check service status every 5 minutes and send the data to OMS to alert and trigger a remediation runbook, so 1 hour would not suffice.  I had recently spoken with a member of the product group around a custom OMS solution and specifically remembered him saying that his runbook was running every 5 minutes….so I was off to investigate!

After a few minutes of searching, I was able to find some good information here.  Although the post referenced the Classic portal, the same basic process still applies and worked in my scenario.  By using a Scheduler Collection we can schedule at a much more granular interval.  I’ve outlined the process in the new portal below.

OMS Alerting and Remediation

A few weeks ago, Microsoft released the OMS Alerting feature in preview, which includes some really useful features like alert notifications and remediation.  We are now able to set up alerts for any saved search query we create in OMS, which gives us the capability to alert on all of our solutions (Security and Audit, Alert Management, custom logs, performance data, etc.).  Additionally, we are given the capability to select Azure Automation runbooks during OMS Alert configuration to remediate our alerts both on-premises and in the cloud.  Very cool!

In my last post, SCOM + OMS + Azure Automation here, I discussed a custom solution using custom fields, the OMS Search API and Azure Automation to automate remediation tasks.  The concepts in this post certainly still apply, but now we have a built in feature which uses webhooks and does not require the OMS Search API logic to be included in our alert remediation runbooks.  Let’s check it out…..

SCOM + OMS + Azure Automation

I recently presented a demo at Infront Consulting Group’s Cloud University outlining a solution using SCOM, OMS, and Azure Automation to automate the process of (1) identifying on-premises servers not currently being monitored in SCOM and (2) remediating the issue by auto-installing agents on those servers. 

Specifically, what we’ll be focusing on in this post are the abilities to collect and write custom logs from your on-premises resources to OMS, use custom fields in OMS to extract a specific value from a line of output (in this example a computer name), and then utilize Azure Automation Runbooks to remediate issues both in the cloud and on-premises from one central location.  If nothing else, you can use this solution as a reference point to get everything configured in your environment to build other cool automation scenarios using OMS and Azure Automation! 

Let’s get started…

OMS Automation Part 2 – Create a Custom Azure Automation Runbook to Execute Against OpsMgr On-Premises

In part 1 of the OMS Automation series I reviewed how to configure the Hybrid Runbook Worker to run Azure Automation against on-premises servers, specifically OpsMgr (here).  Additionally, we covered how to import and configure an existing Azure Automation integration module on the Hybrid Worker agent and in Azure Automation, and then demonstrated how to execute a workflow using the module.  In part 2 of the series, I will dig a little deeper and demonstrate how to create a custom Azure Automation Runbook and execute the workflow against OpsMgr on-premises using the native OpsMgr PowerShell cmdlets.  Let’s get started!

OMS Automation Part 1 – Execute Azure Automation Runbooks Against OpsMgr On-Premises

I was recently tasked with integrating OpsMgr and OMS, in addition to configuring the Automation solution and POC a OpsMgr automation task for a customer. During my adventures locating documentation on the various configuration steps to get everything integrated and working properly, I found that there really is not much “how to” documentation available out there in regards to utilizing the Hybrid Runbook Worker to execute automation against OpsMgr on-premises. SO…sounds like a good opportunity for a blog!

SCOM – Test Console Permissions with PowerShell

Sometimes the simplest solutions are the most effective.  I was on site a few months ago and was asked by a fellow engineer whether I knew of an easy way to test SCOM console permissions without having to log into the local computer as each user.  Sounds like an opportunity to script something up to me!

SCOM – Reset manually closed monitors with PowerShell

Let me start by saying there are several blogs out there with different scripts and methods describing how to do this.  I decided to incorporate a few, add some of my own code and checks, and provide you with a good, in-depth post on how to utilize this invaluable clean up automation.

In most environments where I’ve consulted, one of the first questions I ask when I see a fairly clean active alert view in the console is “have you been closing alerts manually?”, directly followed by the obligatory “do you know the difference between a rule and a monitor?”.  We’ve all heard that question 100 times, right?  If not, you are probably guilty 🙂  I won’t dig too deep into that topic in this post, but here is the high level:  Don’t manually close monitors!

If you do want to dig deeper into how to tell if an alert is generated by a rule or monitor and/or what the differences are, feel free to hit that post here: https://scomanswers.wordpress.com/2015/03/04/scom-rule-vs-monitor/

Back to the topic at hand.  In the aforementioned environments above, in most cases the engineer immediately wants to know how to rectify the issue, and how to address it moving forward.  Depending on how bad the issue is, resetting the environment using maintenance mode works well if done carefully and off-hours,  but preferably, we can use a script to reset manually closed monitors and automate it to moving forward.  Let’s dig into option two!