Unable to stop continuous webjobs

Unable to stop continuous webjobs

Issue:

·       Why webjobs are not stopping? Tried stopping through,

o   Portal

o   Azure CLI

o   REST API

o   PowerShell

Analysis:

This is a common phenomenon and can happens with anyone. We wonder why are we not able to stop webjobs form portal or trough some other approach like azure CLI, REST API etc. while this was tired multiple times in past and it was working fine.

What happened suddenly? 🤔🤔

To understand the reason of this, we first must understand; what happens when we stop webjobs?

Let’s try it. 🤞🤞

To try this let’s create a web job under app service (web app) and then will try stopping it through portal and we'll see if it stops or not.

I just created a web app which doesn’t have any webjobs now.

 Now created a continuous web job which is running successfully

It is very important to check the folder structure of webjobs in SCM aka KUDU site.

To do this, Open the SCM site and navigate to,

D:\home\site\wwwroot\App_Data\jobs\continuous/{webjobs name} path where continuous webjobs is deployed.

Only 2 files are there (.exe and .config).

And now let’s click on stop and see the result.

 

Webjobs stopped successfully.

Now let's observe what happened to folder structure in SCM site.

 

Just observe the difference in number of files before stopping and after stopping the webjobs. Earlier it was 2 and not it is 3.

A new file has been introduced named “disable.job” after stopping the webjobs.

Few points about disable.job file-> This file disappears when we start the webjobs and will appear again when we stop the webjobs. That means,

• Webjobs should have this file when it is stopped.
• Webjobs will also stop even if you place this file manually in that folder.
• Webjobs will start running when you delete this file from your webjobs folder

So, presence of this file is the reason why webjobs is in stopped condition.

Let’s try other approaches.

1.     REST API

Ref: https://docs.microsoft.com/en-us/rest/api/appservice/webapps/stopcontinuouswebjob

Examples post request

POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{name}/continuouswebjobs/{webJobName}/stop?api-version=2019-08-01

 

When a post request was sent for stopping a webjobs through REST API a disable.job file is created (similar to what happened earlier) which means stopping through portal and through REST API does same thing i.e. create a disable.job file inside the D:\home\site\wwwroot\App_Data\jobs\continuous\{webobs name} folder.

 

2.     Azure CLI

Let’s execute the CLI command to stop continuous webjobs.

Ref: https://docs.microsoft.com/en-us/cli/azure/webapp/webjob/continuous?view=azure-cli-latest

Example CLI command

az webapp webjob continuous stop --name MyWebApp --resource-group MyResourceGroup --webjob-name MyWebjob

 

This also resulted same behavior i.e. creating a disable.job file.

Result of analysis

Now picture is clear. Presence of Disable.job is the reason for stopping a webjobs.

This means, when you are trying to stop the webjobs and somehow due to some reason disable.job file is not being created; can be one reason for not able to stop the webjobs.

Now we can focus on scenarios which can cause this issue and one of the most common scenarios is WEBSITE_RUN_FROM_PACKAGE.

When it’s value is set to 1 your WWWROOT folder becomes read-only that means you can’t write any files in WWWROOT folder and disable.job file is written inside that folder only when we perform stop webjobs operation.

With the WEBSITE_RUN_FROM_PACKAGE app setting value of 1, the zip deployment APIs copy your package to the d:\home\data\SitePackages folder instead of extracting the files to d:\home\site\wwwroot

With WEBSITE_RUN_FROM_PACKAGE, if you go into your d:\home\site\wwwroot folder from Kudu Console, you will see all the files from your zip, but the whole folder will be read-only. This is the same view that your runtime will have.

 Reference for WEBSITE_RUN_FROM_PACKAGE,

• https://docs.microsoft.com/en-us/azure/azure-functions/run-functions-from-deployment-package
• https://github.com/Azure/app-service-announcements/issues/84

Above case is only valid when webjobs are deployed inside App_data folder e.g. {D:\home\site\wwwroot\App_Data\jobs\continuous} and have

WEBSITE_RUN_FROM_PACKAGE set to 1 but here we have one more surprise. 😎😎

App_data is not the only folder where you can deploy your webjobs. It was the only folder before introducing the WEBSITE_RUN_FROM_PACKAGE but with the WWWROOT read-only challenges, they have introduced a new location for your webjobs which is

“d:\home\site\jobs\”

Which is outside the WWWROOT folder and inside the site folder.

So now we have 2 scenarios for webjobs deployment,

• those that are deployed with your Web App (under app_data folder)
• those that are deployed separately from your Web App

The first kind of simple, as they will just be part of the zip file, along with the rest of the app. But for those that are deployed separately, things are trickier because they can't just be uploaded to d:\home\site\wwwroot\app_data\jobs\..., which is read-only.

To solve this, Webjobs can now also be deployed to d:\home\site\jobs\.... Jobs from both locations can exist in the same app.

Additional Information

Azure also provide a configuration setting which stops all currently running webjobs.

·       WEBJOBS_STOPPED - Set this setting to 1 to disable running any job (will also stop all currently running jobs)

Adding this setting with value 1 doesn’t create a disable.job file as it stops all currently running webjobs and not belongs to single webjobs.

Be careful when you implement this option.

 

Kill webjobs exe

If issue is very urgent and you want to stop the webjobs then you can  kill the webjobs exe but this should be the last option 

Hope this helps.

Troubleshooting Azure Functions HTTP 404 Errors: How to Fix Them

Troubleshooting Azure Functions HTTP 404 Errors: How to Fix Them

 

Issue:

·       Why Azure functions http trigger returning “HTTP ERROR 404”?

·       I verified my function URL is correct but still 404

Keywords:

• Azure Functions HTTP trigger 404 error
• Troubleshooting Azure Functions 404 issues
• Resolving 404 errors in Azure Functions HTTP triggers
• Debugging Azure Functions HTTP trigger 404s
• Troubleshooting HTTP 404 errors in Azure Functions
• Azure Functions HTTP trigger not found issue
• Fixing Azure Functions HTTP 404 errors
• Azure Functions HTTP trigger routing issues
• Diagnosing Azure Functions HTTP trigger 404 errors

 

Solution:

This is a common phenomenon and can happens with anyone. We wonder what happens to my function app or that function which is returning 404 while my URL is valid.

First of all, what is 404 stands for?

This is an http status which indicates that resource which you are trying to access it not available to serve your http request. In other words, your client like browser, postman, code is able to communicate with server, but server didn’t find the requested resource and returned 404 not found.

Alternative status can be,

• HTTP 404
• 404 not found
• Page not found
• HTTP ERROR 404
• 404 

A simple example from .net is, let’s say your application name is https://www.testingazure.com and you send request for default.aspx page(https://www.testingazure.com/Defalut.aspx ) which is available on server then you will get the response from default.aspx page but if this page is not available in your application the you see 404 not found.

I also encountered this issue and thought to share with everyone.

I created an HTTP trigger function under my function app and when I tried to access it through URL, I received HTTP ERROR 404.

 

I verified following points quickly,

• My function app was up and running.
• HTTP trigger function was available in my function app which I was calling.
• I passed correct code/key in my URL   

What’s wrong then? Why 404?

It was confusing as I was not getting any hints until I realize that through browser you send GET request and what if my function doesn’t support GET request?

I called the same URL through POST method and it worked. It returned the result as expected.

I checked that HTTP trigger function support HTTP Methods and GET was disabled.

 

 

I enabled the GET method and send the same request again this time I received the response successfully.

Similarly, you can verify for other HTTP methods.

You can find and configure this in function.json file also.

 

According to Azure function app official documents, HTTP Methods property in azure function is “An array of the HTTP methods to which the function responds. If not specified, the function responds to all HTTP methods.”

This means, not only resource availability but also how you are accessing it can cause the HTTP 404.

So, whenever you observe HTTP 404 with your http trigger function please validate following,

• Is your function app up and running?
• Is HTTP trigger function available in your function app which you are calling?
• Are you passing correct code/key in my URL?
• This will return HTTP ERROR 401 but better to check safe side
• Does your function support http method which you are using? 

Hope this helps. Please let me know if you have any query.

 

WebJob cannot be added from portal if deployment from source control is configured

WebJob cannot be added from portal if deployment from source control is configured.

Issue:

·       WebJob cannot be added from portal if deployment from source control is configured.

 

Solution:

Azure webjobs is very popular now a days. It is used to run background task what we used to do with window schedular and here we are discussing about a common phenomenon which can be faced by anyone while create webjobs from the portal.

You can check in below screen show what happens when I click on Add button for creating a new web job.

webjobs

WebJob cannot be added from portal if deployment from source control is configured.

This message is clear, and we can easily decode it. This message indicates that your app service has some source control configured for CI/CD e.g. GitHub, DevOps etc and you can validate this in the deployment center option of your app service; like in my case I have configured GitHub as source control which is preventing me to me add new webjobs.

 

 

Now we know why we are not able to add webjobs? Then how to solve this issue.

If you have basic knowledge of webjobs then you should know the location of webjobs folder after deployment.

There are various ways to access webjobs folders and few of them are,

• SCM or KUDU site
• FTP

The path for webjobs is,

• Triggered web jobs:

            d:\home\site\wwwroot\app_data\jobs\triggered\{job name}

• Continuous webjobs:

       d:\home\site\wwwroot\app_data\jobs\continuous\{job name}

Note: While if you have Website_Run_From_Package enable (which makes d:\home\site\wwwroo folder read-only) then webjobs path will be,

• d:\home\site\jobs\

That means you can directly deploy a WebJob in that path and your webjobs will start appearing in the portal.

This action will work with both SCM/KIDU and FTP.

Important:

You must be feeling happy that you are able to create webjobs, but this is not the permanent solution. This is just to unblock you if you want to create webjobs on urgent basis.

The error message was there for some reason which means If you have source control configured with your application, the webjobs should be deployed as part of the source control integration as webjobs is the part of your webapp.

If you deployed webjobs separately and then you deploy your webapp from the source control with “Remove additional files at destination” option checked then your webjobs will be deleted as they are not available in your deployment package under source control and you will be in trouble.

The correct way to deploy your webjobs in this case is to integrate them with your webapp in source control so that they are also deployed along with your app service.

Hope this helps. Please let me know if you have any query in this regard.

Choose consumption plan name

Choose consumption plan name in Azure function app

Create consumption plan with your desire name

 

Issue:

·       How can I create consumption plan with my desire name like App Service plan?

 

Solution:

Unlike App service plan or premium plan, Azure portal doesn’t provide you a way to create a consumption plan with your desire name. You can’t decide consumption plan name at the time of creating the Azure function.

You can only select the plan type as mentioned in below screen.

 

It creates consumption plan name by its own mostly with some random name, like in below screen,

While if you choose App Service plan or Premium plan then you can create hosting plan with your desire name also. You can check same in below screen,

Similarly, in Premium plan you can create with your desire name,

 

 

Now question should be, why should I worry about consumption plan name? I can use the default name also.

And the answer is, in some cases Naming Conventions matters. For example, suppose you have 3 function app for 3 environments like DEV, UAT and PROD and you want to have consumption plan name accordingly like consumptoinDEV, consumptionUAT or consumptionPROD.

As mentioned earlier, you can’t decide the consumption plan name when create through portal then how to achieve it.

Answer is: ARM Template.

 

 

You can check in above screen that ARM template provides you a way to create consumption plan with your own desire name.

Complete ARM template for function app in consumption plan : https://github.com/Azure/azure-quickstart-templates/tree/master/101-function-app-create-dynamic

 

Now I deployed function app using ARM template and specified my own name and here is the result,

Hope this helps. Please let me know your queries in comment section.

Happy to help. 😊

Azure Functions timeout

Everything about function app timeout

 

Issue:

·       Timeout value of 00:05:00 exceeded by function?

·       What is azure functions timeout?

·       What is azure functions execution timeout?

·       What is default timeout in azure function?

·       Consumption plan timeout?

·       Premium plan timeout?

 

Solution:

What is timeout?  In business logic language, a timeout is a specified period of time that will be allowed to complete an operation and if that operation doesn’t complete within specified period of time then it will throw some exception. Timeouts allow for more efficient usage of limited resources without requiring additional interaction from the agent interested in the goods that cause the consumption of these resources.

A common example is your smart phone turn off backlight after a certain time when idle which you can set.

Similarly, Azure has set the timeout for function app and it is very important to understand it as you can develop/write your business logic and optimize it accordingly.

In Azure functions, timeout means if your business logic or code should be optimized enough to complete with timeout duration otherwise it throws exception like “Timeout value of 00:05:00 exceeded by function”

In Azure functions, timeout varies according to hosting plan and runtime versions.

Azure functions have 3 hosting plans,

• Consumption plan (Dynamic plan)
• Premium plan (Elastic premium plan)
• App Service plan (Dedicated plan

And as of now it has 3 runtime versions,

·       1.x

·       2.x

·       3.x

Where 1,2 and 3 is the major version and x stand for minor version e.g. 2.09234.

Now we know what azure functions execution timeout is? hence we should focus on

·       Default timeout,

·       maximum timeout

for each hosting plan.

You can check this information in the below list.

That means if you see exception like “Timeout value of 00:05:00 exceeded by function” then you quickly need to check if your function is hitting the default timeout.

Here you have 2 options,

1.      Investigate why your code execution is taking that much of time and optimize your code

2.      Increase the default timeout.

For first case you better know how to optimize the code.

How to modify the default timeout

Azure functions timeout value is stored and handled in host.json file and attribute name is functionTimeout

{     "functionTimeout": "00:05:00" }

 

functionTimeout indicated the timeout duration for all the functions and not for the single function.

You can update the default timeout value, save it, and test your function again.

In dedicated app service plan there is no limit but you should keep a fixed upper bound limit as unlimited timeout can have some disadvantages like your execution will keep executing and you will have to pay for it.

For unlimited timeout you can use -1

{

    "functionTimeout": "-1"

}

Important note

Regardless of the function app timeout setting, 230 seconds is the maximum amount of time that an HTTP triggered function can take to respond to a request. This is because of the default idle timeout of Azure Load Balancer. For longer processing times, consider using the Durable Functions async pattern or defer the actual work and return an immediate response.

Function timeout at function level

We know now that we can set function timeout in host.json but it is for all the function inside it. What about setting timeout for one or 2 function only.

While developing the functions from Visual Studio, you can now set the timeout at the function level.

Adding the attribute [Timeout("00:00:10")], above the function definition would set the timeout value for the function.  more details.

I did test the function executions to confirm that it doesn’t affect executions of other functions in the same function app.

With the test deployment I made – I deployed multiple function to the Function App on Azure, with only one of them (say function A) having the timeout set. While executing the functions, we see that A stops after the threshold timeout value and the others continue as expected.

Imp Note: This is for C# language only.

Output:

Hope this helps. Please let me know if you have any query

References:

https://docs.microsoft.com/en-us/azure/azure-functions/functions-scale

https://docs.microsoft.com/en-us/azure/azure-functions/functions-host-json#functiontimeout