SQLServerPedia

SQL Server 2005, plan cache, becoming bloated with excessive amounts of adhoc queries with a usecount of 1, crashing the server.  64 Bit SQL with 32gb of RAM.

Ran some queries to determine which database was causing the most adhoc cache plan bloat, than we set the force auto-parameterization on.

-- Get the size of the Plan Cache (CACHESTORE_SQLCP is non-SP and CACHESTORE_OBJCP is SP)

SELECT (SUM(single_pages_kb) + SUM(multi_pages_kb) ) / (1024.0 * 1024.0) AS 'Plan Cache Size(GB)'

FROM sys.dm_os_memory_cache_counters

WHERE type = 'CACHESTORE_SQLCP'

OR type = 'CACHESTORE_OBJCP'

-- UseCounts and # of plans for Adhoc plans

SELECT usecounts, count(*) as no_of_plans

FROM sys.dm_Exec_Cached_plans

WHERE cacheobjtype = 'Compiled Plan'

AND objtype = 'Adhoc'

GROUP BY usecounts

ORDER BY usecounts

Use this DMV query to find the offenders:

-- Find the ad-hoc queries that are bloating the plan cache

SELECT top 1000 *

FROM sys.dm_Exec_cached_plans

CROSS APPLY sys.dm_exec_sql_text(plan_handle)

WHERE cacheobjtype = 'Compiled Plan'

AND objtype = 'Adhoc' AND usecounts = 1

--AND size_in_bytes < 200000

ORDER BY size_in_bytes DESC

-- Setting the PARAMETERIZATION option to FORCED

ALTER DATABASE [adtempus_DTSR] SET PARAMETERIZATION FORCED WITH NO_WAIT

GO

-- Setting the PARAMETERIZATION option to SIMPLE (default)

ALTER DATABASE [adtempus_DTSR] SET PARAMETERIZATION SIMPLE WITH NO_WAIT

GO

-- The current setting of this option can be determined by examining

-- the is_parameterization_forced column in the sys.databases catalog view.

I was perusing the MSSQL 2008 site and I ran across the SQL Server store. Where are the tshirts and hats? Yes, that is how I roll.

Anyway, they do have the learning materials(HOL) at prices that cannot be beat.

One of the most commonly asked questions on the AS MSDN Forum is how to format measures that contain values in different currencies with the correct currency symbol. I've never blogged about this because a lot of people have already written up the solution in detail, for example Mosha:
http://www.sqljunkies.com/WebLog/mosha/archive/2005/10/13/mdx_format_currency.aspx

and Vidas:
http://www.ssas-info.com/analysis-services-faq/27-mdx/244-how-change-currency-symbol-based-on-selected-currency-dimension-member

However I was thinking about this recently and in my opinion there's a big problem with using the Language property to do this. And that is that when you set the Language of a cell, you not only change the currency symbol but you also change other ways that the number is formatted, for example the symbols used as thousands separators and decimal separators. In the US and UK of course, we use full stops (I think they're called 'periods' in the US?) as decimal separators and commas as thousands separators, but in continental Europe the roles are reversed. So the value:
100,001
would be interpreted as one hundred thousand and one in the UK, but one hundred point zero zero one in Germany, say, and the value:
100.001
would be interpreted in the opposite way. Borrowing one of the screenshots from Vidas's post you can see how the Language property respects these conventions:

So you can see what the potential problem is - what happens if you have values in Euros, USDs and GBPs in your cube? However much you educate your users you can guarantee that someone at some time is going to get confused or worse not realise what's going on and interpret the values incorrectly.

What's the alternative then? I think using Format_String has to be the way to go. If you alter Vidas's example so that instead of locale ids you put currency symbols inside the currency dimension named query, for example:

SELECT     CurrencyKey, CurrencyAlternateKey, CurrencyName,
                      CASE CurrencyAlternateKey WHEN 'GBP' THEN '£' WHEN 'EUR' THEN '€' WHEN 'JPY' THEN '¥' WHEN 'USD ' THEN '$ ' END AS LocaleID
FROM         DimCurrency
WHERE     (CurrencyKey IN
                          (SELECT DISTINCT CurrencyKey
                            FROM          FactCurrencyRate))

and then change his MDX assignment to be something like:

SCOPE ([Destination Currency].[Destination Currency].[Destination Currency].Members);
Format_String(This) = [Destination Currency].[Destination Currency].[Symbol].MemberValue + "#,#.00";
END SCOPE;

Then you get the desired result. However, one thing I did notice when I was experimenting with this is that if you try to use more than one character for your symbol (for example you might want to use CHF for Swiss Francs) you sometimes get the following error:

#Error The following system error occurred:  Out of present range. .

Not good. Here's the bug logged on Connect:
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=339913

Following on from all this, it also makes sense that users connecting from different locales automatically see numbers (but not currency symbols) formatted in the convention of their locale. So a German person might connect to the cube and see Euros with a € and USDs with a $, but see commas used as decimal separators, whereas a user in the UK would still see €s and $s with the correct symbol but full stops used as decimal separators. Now AS2K I seem to remember used to be able to handle this perfectly well - it could display the appropriate decimal separator and thousand separator depending on the client locale. However AS2005 RTM had a problem in that it worked ok for calculated measures but not for real measures; this was 'fixed' in SP2 so both calculated measures and real measures always got displayed in the locale of the server. American software, eh? And to think that so many members of the dev team are from Europe too. Here's the Connect:
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=218858

and here's a forums thread on the subject:
http://forums.microsoft.com/msdn/showpost.aspx?postid=1488729&siteid=1&sb=0&d=1&at=7&ft=11&tf=0&pageid=1

Hmmmmm.... I need to check if this has been fixed properly in Katmai.

First things first – depending on the version of SQL Server that you’re using, SQL Server could be creating this problem for itself. See these articles if you’re running SQL Server 2005 pre-SP2:

• SQL Server becomes sluggish or appears to stall on 64 bit installations
• A significant part of sql server process memory has been paged out. This may result in performance degradation
• How to reduce paging of buffer pool memory in the 64-bit version of SQL Server 2005
• The SQL Server Working Set Message

So what’s going on? First we need to address two concepts: Virtual Address Space and Working Set. Windows uses virtual address space (VAS) to allocate memory to applications (that’s as deep as I’m going here). A program’s working set is the collection of VAS pages that have been recently referenced. When a process is created, it is assigned a minimum and maximum working set size. As the working set size increases, thereby increasing memory demand, the virtual memory manager will attempt to keep up to the maximum working set size (memory) resident without exceeding the maximum threshold.

One common misconception is that there is as much virtual memory available to a process as there is physical memory (RAM) on a machine. Not so (or I wouldn’t be going there); you can get much more depth by reading Slava OKs’s blog on VAS. One very important point from a SQL Server perspective, however, is that when you’re configuring min and max server memory, you have to take into account the MemToLeave area. MemToLeave is VAS that’s left unused when SQL Server starts so that external components have address space to consume when they’re loaded. Typically, the majority of memory allocations from MemToLeave are from non-SQL Server memory consumers like COM objects, extended stored procedures, and linked servers. You can use DBCC MEMORYSTATUS to see these, but bear in mind that what’s reported will only reflect allocations made by SQL Server directly (SQL Server has no knowledge of outside memory requests).

So, when troubleshooting out of memory errors, do your homework.

• Check the list above to see whether your SQL Server is suffering from a known SQL Server bug, and understand (at least a little bit about) VAS and memory allocations.
• You can use Performance Monitor (Task Manager won’t help here) to get an idea of how much of VAS is currently consumed inside of your process by looking at the virtual bytes counter.
• If you believe that a 3rd party application is loading DLLs into SQL Server’s VAS, or you want to see if your own CLR or external procedures are doing so use SQL Server’s DMV sys.dm_os_loaded_modules to see what’s been loaded into SQL Server’s address space.

Oh, and by the way, for anyone who’s used to configuring the working set size parameter with sp_configure, you need to be aware that it’s been deactivated in SQL Server 2005.

Hope that helps.

One of the most commonly asked questions on the AS MSDN Forum is how to format measures that contain values in different currencies with the correct currency symbol. I've never blogged about this because a lot of people have already written up the solution in detail, for example Mosha:
http://www.sqljunkies.com/WebLog/mosha/archive/2005/10/13/mdx_format_currency.aspx

and Vidas:
http://www.ssas-info.com/analysis-services-faq/27-mdx/244-how-change-currency-symbol-based-on-selected-currency-dimension-member

However I was thinking about this recently and in my opinion there's a big problem with using the Language property to do this. And that is that when you set the Language of a cell, you not only change the currency symbol but you also change other ways that the number is formatted, for example the symbols used as thousands separators and decimal separators. In the US and UK of course, we use full stops (I think they're called 'periods' in the US?) as decimal separators and commas as thousands separators, but in continental Europe the roles are reversed. So the value:
100,001
would be interpreted as one hundred thousand and one in the UK, but one hundred point zero zero one in Germany, say, and the value:
100.001
would be interpreted in the opposite way. Borrowing one of the screenshots from Vidas's post you can see how the Language property respects these conventions:

So you can see what the potential problem is - what happens if you have values in Euros, USDs and GBPs in your cube? However much you educate your users you can guarantee that someone at some time is going to get confused or worse not realise what's going on and interpret the values incorrectly.

What's the alternative then? I think using Format_String has to be the way to go. If you alter Vidas's example so that instead of locale ids you put currency symbols inside the currency dimension named query, for example:

SELECT     CurrencyKey, CurrencyAlternateKey, CurrencyName,
                      CASE CurrencyAlternateKey WHEN 'GBP' THEN '£' WHEN 'EUR' THEN '€' WHEN 'JPY' THEN '¥' WHEN 'USD ' THEN '$ ' END AS LocaleID
FROM         DimCurrency
WHERE     (CurrencyKey IN
                          (SELECT DISTINCT CurrencyKey
                            FROM          FactCurrencyRate))

and then change his MDX assignment to be something like:

SCOPE ([Destination Currency].[Destination Currency].[Destination Currency].Members);
Format_String(This) = [Destination Currency].[Destination Currency].[Symbol].MemberValue + "#,#.00";
END SCOPE;

Then you get the desired result. However, one thing I did notice when I was experimenting with this is that if you try to use more than one character for your symbol (for example you might want to use CHF for Swiss Francs) you sometimes get the following error:

#Error The following system error occurred:  Out of present range. .

Not good. Here's the bug logged on Connect:
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=339913

Following on from all this, it also makes sense that users connecting from different locales automatically see numbers (but not currency symbols) formatted in the convention of their locale. So a German person might connect to the cube and see Euros with a € and USDs with a $, but see commas used as decimal separators, whereas a user in the UK would still see €s and $s with the correct symbol but full stops used as decimal separators. Now AS2K I seem to remember used to be able to handle this perfectly well - it could display the appropriate decimal separator and thousand separator depending on the client locale. However AS2005 RTM had a problem in that it worked ok for calculated measures but not for real measures; this was 'fixed' in SP2 so both calculated measures and real measures always got displayed in the locale of the server. American software, eh? And to think that so many members of the dev team are from Europe too. Here's the Connect:
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=218858

and here's a forums thread on the subject:
http://forums.microsoft.com/msdn/showpost.aspx?postid=1488729&siteid=1&sb=0&d=1&at=7&ft=11&tf=0&pageid=1

Hmmmmm.... I need to check if this has been fixed properly in Katmai.

SQL Server 2005, Cumulative Update 7.

We downloaded and applied for testing, all tested well.

We tried to implement, and ran into a problem with a vendor install for Symantec End Point reporting database.

We could not figure out the issue, the vendor recommended reverting back to SQL Server 2005, without CU7.  Installation completed succesfully.

We never fullly figured out what was in the vendor script that caused it to fail under CU7 and work in CU6, but obviously something was different.

Based on this "unknown" factor we have not standardized on CU7, in the past none of the CU's have ever broken the functionality of an application, but something in CU7 sure does with Symantec End Point reporting.

If you’ve ever taken the time to read http://www.tpc.org/results/FDR/TPCC/hp_orca1tb_win64_fdr.pdf, you might find a bit in there that’s confusing. “Microsoft SQL Server 2005 Enterprise Edition 64-bit Service Pack 1 was configured to utilize “soft NUMA”, a feature that allows network connections to be affined to specific groups of CPUs (this is independent of the hardware NUMA features of the HP Integrity  Superdome). SQL Server was configured with 33 SoftNuma nodes. 31 of these nodes were configured with 2 CPUs each. The 32nd SoftNuma node as configured with one CPU and used for the checkpoint process. The 33rd SoftNuma node, also with one processor, was reserved for the Log Writer thread. A script doing continuous checkpoints of 1700 seconds (28 minutes, 20 seconds) was started on one of the server after steady state was reached with a connection port that connected to the 33rd Numa Node.  This allowed the main checkpoint process to run on that single processor, which directed the processing of the checkpoint tasks that SQL assigned to each of the other 31 SoftNuma Nodes.” 

What’s confusing about this is it’s reference to the checkpoint process, since the BOL also says that you can configure I/O affinity and CPU but not specific threads.

 

As it turns out, this issue is explained in the BOL article “How to” Map TCP/IP Ports to NUMA Nodes”.  The “connection process” mentioned above is not the more commonly know background checkpoint that flushes dirty pages to disk, but is a separate connection affinitized (is that even a word?) to the 32node which issues a CHECKPOINT command. 

 

Thanks to Eladio Rincon (Solid Quality Mentors) and Peter Byrne (Microsoft) for working this one out.

I've seen two isolated incidents on SQL Server 2005 where restoring from a SQL Lite Speed backup where the MDF, NDF and LDF file permissions were changed to the user who performed the restore. 

Everthing works fine, but then in the future we decided to move these files, but to our surprise were were unable to move them, giving us an error that the files were in use, read-only or did not have permissions.  We spent considerable time looking for what process had the files in use (Virus Scan, netbackup, SQL ? (process and file explorer from sysinternals), and finally read the error again and decided, maybe the files are read-only !  In the process of checking this, we clicked on the security tab, and the individual who performed the restore was the only account with permissions to the files !

We changed the permissions and copied them fine.

I don't know if this was caused by Polyserve, Lite-Speed or SQL Server; but definetly caused us some frustration, very strange indeed !

More: Since there are 3 machines in this mix so I’m not sure which one I should be checking. Computer A is running Windows 2000 and SQL Server 2000; computer B is running Windows XP Professional and Oracle Server 10.2.0.2; computer C is running Windows XP Professional with SQL Server 2005 client tools (Management Studio) and Oracle 10.2.0.2 client. I am using Computer C to transfer data from Computer A to Computer B. To confuse me even more, using computer D, which has SQL Server 2000 client tools, I can transfer the data from Computer A to Computer B with no problem. Doesn’t that mean A and B are configured correctly?

You need to address computer C in the scenario you outlined as that version of MDAC/ODBC/Oracle client is being used. There is likely a mismatch between the SQL Server 2005 tools and your linked server. I would try the following:
1) ensure computer C is running the latest version of MDAC and an ODBC driver that fully supports Oracle 10g
2) check that you’ve got the latest version of the Oracle client on the linked SQL Server machine. Some customers have said that they require patch 5203839
3) ensure you’ve configured the connection to “Allow inprocess” in the provider options

Finally, check out these MSDN links to ensure your Microsoft software is completely up-to-date for this scenario:
http://support.microsoft.com/kb/280106
http://msdn2.microsoft.com/en-us/library/ms190618.aspx
http://msdn2.microsoft.com/en-us/library/ms189063.aspx

Hope that helps.

One of the best things about becoming a Microsoft MVP is meeting other MVPs.  I bring this up because last week was the annual MVP Summit in Seattle, WA.  I was really looking forward to meeting Steve Kass.  Steve Kass is one of the smartest SQL Server MVPs I’ve encountered, especially when it comes to SQL questions.  A while back, I noticed that Steve made an interesting recommendation for a hash function that you could use for partitioning that I thought was worth noting.  A hash function would be very useful if you wanted to implement your own variation of a range partition using a hash function rather than the standard sort of range partitioning where colA values of A-H go to partition 1, values of I-P got to partition 2, and so forth.

 

Steve notes that you could use the following for hashing something small in size:

   CAST(    SUBSTRING(      HASHBYTES(‘SHA1′,         CAST(my_col AS NVARCHAR(appropriate_size))),8,1) AS tinyint) 

This is just an off the cuff recommendation from Steve and might need some fine tuning, for example, the CAST might throw off persistence.  However, it’s a good start.

 

Thanks, Steve, for sharing this and thanks, readers, for sharing any improvements you might develop out in the field.