• +44 (0)20 3772 1081
  • sales@sqlmantratools.com

Partial Key Filtering Issue in Dynamics NAV and Business Central

Introduction to performance tuning in Dynamics NAV and Business Central

Recently, one of our customers called our support desk asking for emergency help in identifying a severe performance issue with their live Business Central site. The web service was consuming 100% CPU, and once it hit that level, it remained at 100%, eventually locking to the extent that no one was able to use the system at all.

The customer’s infrastructure team allocated more and more CPU to the service tier machine and discovered that this only delayed the web service reaching 100% CPU rather than solving the issue. As a result, they could not trade. Their system was down during their peak trade. Due to the severity of the problem, the case was escalated to one of our senior performance tuning experts.

The customer and their IT team were clueless about the root cause of the issue. The only thing they could recall was that they had recently deployed quite a substantial enhancements to their website, as well as changes to the way it integrates with their Business Central system.

The customer’s CEO stated that they tested all these changes in a test environment and everything worked fine. They said they had never seen such bad performance issues and questioned what had happened.

Our senior consultant replied "this is quite common. You are never able to properly stress test (performance test) a system in a test environment, as it is not realistic to simulate the concurrent workload that a live system goes through. That is why having a proper performance monitoring tool, such as SQL Mantra Tool’s Performance Insights performance monitoring tool, is essential for all businesses".

In this case, having SQL Mantra Tool paid off for the client. SQL Mantra Tool had been passively recording system activity and worked like a "black box"., capturing all the necessary details for a performance tuning exercise.

Many of you may be interested to know what the issue was and how we fixed it. Let’s dive into the problem.

We turned to SQL Mantra Tool’s automated performance monitoring feature, and thanks to its unique ability to capture application code (A/L and C/AL), we managed to diagnose the issue with pinpoint accuracy in no time.

The issue was caused by the way tables such as Sales Header, Sales Header Archive, and Sales Line Archive were filtered. Their e-commerce website was calling a web service in Business Central, passing information such as the sales order number along with other data. In Business Central, the following code was used to process that request:

                
                  local procedure ProcessOrder(pOrderNumber: Text)
                  var
                    SalesHeader: Record "Sales Header";
                    SalesLine: Record "Sales Line";
                    SalesHeaderArchive: Record "Sales Header Archive";
                    SalesLineArchive: Record "Sales Line Archive";
                  begin
                    SalesHeader.Setrange("No.", pOrderNumber);
                    If SalesHeader.Findset() then
                    Repeat
                      // Processing Code
                      SalesLine.Setrange("Document No.", SalesHeader."No.");
                      If SalesLine.Findset() then
                      Repeat
                        // Processing Code
                      Until SalesLine.Next = 0;

                      SalesHeaderArchive.Setrange("No.", SalesHeader."No.");
                      If SalesHeaderArchive.Findset() then
                      Repeat
                        // Some Processing Code
                        SalesLineArchive.Setrange("Document No.", SalesHeaderArchive."No.");
                        If SalesLineArchive.Findset() then
                        Repeat
                          // Some Processing Code
                        Until SalesLineArchive.Next = 0;
                      Until SalesHeaderArchive.Next = 0;
                    Until SalesHeader.Next = 0;
                  end;
                
              

There is no functional issue with the above code; however, it is not written with performance in mind. The Sales Header, Sales Line, Sales Header Archive, and Sales Line Archive tables all have composite primary keys.

When filtering tables that have composite primary keys, it is important to filter on fields that are part of the primary key, especially the fields at the beginning of the key. For example, the primary key of the Sales Header table is defined as "Document Type, No.". When filtering the Sales Header table to retrieve an order, it is essential to filter on the "Document Type". field as well as the "No.". field.

The same applies to the Sales Line, Sales Header Archive, and Sales Line Archive tables. If only one part of a composite primary key is filtered, especially when the leading key field is not included, the system will perform a table scan and will not utilise the index. The larger the table, the slower the query becomes.

Partial key filtering issue occurs when a table with a composite primary key is filtered without including the leading key fields, forcing SQL Server to perform a table scan instead of using an index in Business Central and Dynamics NAV system.

In this processing code, records are updated based on business logic. As a result, the second and subsequent SQL queries are issued with update table lock hints. After a few executions, SQL Server begins lock escalation, and very quickly, business critical tables such as Sales Header and Sales Line will have table level locks. This severely affects system concurrency, leaving users especially in sales and warehousing unable to work.

Once the problem was correctly diagnosed (thanks to SQL Mantra Tool), the fix was simple. We added a few additional Setrange commands (highlighted below). Within an hour, the customer was able to trade again, handle their peak workload, and start shipping goods to meet delivery deadlines.

Before this incident, the CEO had questioned the need for a performance monitoring tool and tried to save a few pounds (roughly the price of a coffee). After this experience, he was convinced it was worth the investment to de risk key business operations and it may well have saved his job.

Please find the tuned code below. Changes are highlighted in bold.

                
                  local procedure ProcessOrder(pOrderNumber: Text)
                  var
                    SalesHeader: Record "Sales Header";
                    SalesLine: Record "Sales Line";
                    SalesHeaderArchive: Record "Sales Header Archive";
                    SalesLineArchive: Record "Sales Line Archive";
                  begin
                    SalesHeader.Setrange("No.", pOrderNumber);
                    SalesHeader.Setrange("Document Type", SalesHeader."Document Type"::Order); 
                    If SalesHeader.Findset() then
                      Repeat
                        // Processing Code
                      SalesLine.Setrange("Document No.", SalesHeader."No.");
                      SalesLine.Setrange("Document Type", SalesHeader."Document Type");
                      If SalesLine.Findset() then
                      Repeat
                        // Processing Code
                      Until SalesLine.Next = 0;

                      SalesHeaderArchive.Setrange("No.", SalesHeader."No.");
                      SalesHeaderArchive.Setrange("Document Type", SalesHeader."Document Type");
                      If SalesHeaderArchive.Findset() then
                      Repeat
                        // Some Processing Code
                        SalesLineArchive.Setrange("Document No.", SalesHeaderArchive."No.");
                        SalesLineArchive.Setrange("Document Type", SalesHeaderArchive."Document Type");
                        If SalesLineArchive.Findset() then
                        Repeat
                          // Some Processing Code
                        Until SalesLineArchive.Next = 0;
                      Until SalesHeaderArchive.Next = 0;
                    Until SalesHeader.Next = 0;
                  end;
              
            
SQLMantraTools Logo

SQL Mantra Tools will monitor Dynamics NAV & Business Central SaaS Performance in Microsoft cloud 24/7 and will give insights to pinpoint the performance pain points. Our software will give the A/L & CAL code as well as the BC & NAV user details along with the SQL Code which was causing the performance issues in both SaaS and On-Prem platforms . This is the unique feature of our software compared to many others.


SQL Mantra Tools will also analyse Microsoft SQL Server key setup parameters as well as its resources along with application setups and will check them against the best practice guidelines and will suggest the necessary steps to remedy them.


Our Maintenance routine will proactively maintain your ERP system, such as Dynamics NAV & Business Central, free from any performance issues, optimising its size, minimising the database growth.