Unable to stop continuous webjobs

Unable to stop continuous webjobs

Issue:

·       Why webjobs are not stopping? Tried stopping through,

o   Portal

o   Azure CLI

o   REST API

o   PowerShell

Analysis:

This is a common phenomenon and can happens with anyone. We wonder why are we not able to stop webjobs form portal or trough some other approach like azure CLI, REST API etc. while this was tired multiple times in past and it was working fine.

What happened suddenly? 🤔🤔

To understand the reason of this, we first must understand; what happens when we stop webjobs?

Let’s try it. 🤞🤞

To try this let’s create a web job under app service (web app) and then will try stopping it through portal and we'll see if it stops or not.

I just created a web app which doesn’t have any webjobs now.

 Now created a continuous web job which is running successfully

It is very important to check the folder structure of webjobs in SCM aka KUDU site.

To do this, Open the SCM site and navigate to,

D:\home\site\wwwroot\App_Data\jobs\continuous/{webjobs name} path where continuous webjobs is deployed.

Only 2 files are there (.exe and .config).

And now let’s click on stop and see the result.

 

Webjobs stopped successfully.

Now let's observe what happened to folder structure in SCM site.

 

Just observe the difference in number of files before stopping and after stopping the webjobs. Earlier it was 2 and not it is 3.

A new file has been introduced named “disable.job” after stopping the webjobs.

Few points about disable.job file-> This file disappears when we start the webjobs and will appear again when we stop the webjobs. That means,

• Webjobs should have this file when it is stopped.
• Webjobs will also stop even if you place this file manually in that folder.
• Webjobs will start running when you delete this file from your webjobs folder

So, presence of this file is the reason why webjobs is in stopped condition.

Let’s try other approaches.

1.     REST API

Ref: https://docs.microsoft.com/en-us/rest/api/appservice/webapps/stopcontinuouswebjob

Examples post request

POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{name}/continuouswebjobs/{webJobName}/stop?api-version=2019-08-01

 

When a post request was sent for stopping a webjobs through REST API a disable.job file is created (similar to what happened earlier) which means stopping through portal and through REST API does same thing i.e. create a disable.job file inside the D:\home\site\wwwroot\App_Data\jobs\continuous\{webobs name} folder.

 

2.     Azure CLI

Let’s execute the CLI command to stop continuous webjobs.

Ref: https://docs.microsoft.com/en-us/cli/azure/webapp/webjob/continuous?view=azure-cli-latest

Example CLI command

az webapp webjob continuous stop --name MyWebApp --resource-group MyResourceGroup --webjob-name MyWebjob

 

This also resulted same behavior i.e. creating a disable.job file.

Result of analysis

Now picture is clear. Presence of Disable.job is the reason for stopping a webjobs.

This means, when you are trying to stop the webjobs and somehow due to some reason disable.job file is not being created; can be one reason for not able to stop the webjobs.

Now we can focus on scenarios which can cause this issue and one of the most common scenarios is WEBSITE_RUN_FROM_PACKAGE.

When it’s value is set to 1 your WWWROOT folder becomes read-only that means you can’t write any files in WWWROOT folder and disable.job file is written inside that folder only when we perform stop webjobs operation.

With the WEBSITE_RUN_FROM_PACKAGE app setting value of 1, the zip deployment APIs copy your package to the d:\home\data\SitePackages folder instead of extracting the files to d:\home\site\wwwroot

With WEBSITE_RUN_FROM_PACKAGE, if you go into your d:\home\site\wwwroot folder from Kudu Console, you will see all the files from your zip, but the whole folder will be read-only. This is the same view that your runtime will have.

 Reference for WEBSITE_RUN_FROM_PACKAGE,

• https://docs.microsoft.com/en-us/azure/azure-functions/run-functions-from-deployment-package
• https://github.com/Azure/app-service-announcements/issues/84

Above case is only valid when webjobs are deployed inside App_data folder e.g. {D:\home\site\wwwroot\App_Data\jobs\continuous} and have

WEBSITE_RUN_FROM_PACKAGE set to 1 but here we have one more surprise. 😎😎

App_data is not the only folder where you can deploy your webjobs. It was the only folder before introducing the WEBSITE_RUN_FROM_PACKAGE but with the WWWROOT read-only challenges, they have introduced a new location for your webjobs which is

“d:\home\site\jobs\”

Which is outside the WWWROOT folder and inside the site folder.

So now we have 2 scenarios for webjobs deployment,

• those that are deployed with your Web App (under app_data folder)
• those that are deployed separately from your Web App

The first kind of simple, as they will just be part of the zip file, along with the rest of the app. But for those that are deployed separately, things are trickier because they can't just be uploaded to d:\home\site\wwwroot\app_data\jobs\..., which is read-only.

To solve this, Webjobs can now also be deployed to d:\home\site\jobs\.... Jobs from both locations can exist in the same app.

Additional Information

Azure also provide a configuration setting which stops all currently running webjobs.

·       WEBJOBS_STOPPED - Set this setting to 1 to disable running any job (will also stop all currently running jobs)

Adding this setting with value 1 doesn’t create a disable.job file as it stops all currently running webjobs and not belongs to single webjobs.

Be careful when you implement this option.

 

Kill webjobs exe

If issue is very urgent and you want to stop the webjobs then you can  kill the webjobs exe but this should be the last option 

Hope this helps.