Virtual Memory Optimization Guide!

[Scarceas]

I think I read it in the forums at Storagereview.com, but it's been ages ago and I don't have a link to the particular post.

It was an interesting enough idea that it stuck in the back of my mind, and when I was reading your guide I remembered it, and wondered if it was important or not... Guess not?

[Adrian Wong]

Well, the trouble with statements made by people in the forums is that anyone can say anything.

That's the reason why we prefer to write and post articles. This allows people to critique. We can then refine it.

In the end, we hope to present definitive conclusions on the particular topic, instead of thousands of "opinions" on the same topic.

[arth1]

The article repeating a common error in stating that on modern systems with lots of ram, you need less paging space. If you're a developer, or have production systems with support, you almost certainly want to be able to save a paging file dump if the system crashes. That requires that your paging file is large enough to hold a system memory dump. Due to what's similar to a file header in a swap file, that means you need at least 1xRAM+1 block of swap.
For users who neither develop nor receive support on their system which might find a full dump useful, they obviously don't have this requirement. But many of us do, and it's a disservice to ignore this.

As for using multiple paging files on multiple drives, Windows NT/2000 will automatically stripe when using two or more paging files (or rather, always write a page to the lowered numbered paging file with the most space free, which amounts to striping in practice). Just set the max size to the same size, and you'll get a striping effect. For XP, the same holds true, but if one drive is busy, the next drive will be immediately tried, meaning for single-block pages, XP is often faster, but for large pages, it evens out.

Finally, make sure that the block size on the drive is equal to or a multiple of the ram page size. Unless running weird hardware (like NT 3.5 on RISC), the RAM page size will normally be 4k, so choosing a 4k block size while formatting the drive will be ideal, while 8k/16k/32k/64k also works, but won't make for optimal "striping" if using multiple paging files on multiple drives.

Regards,
--
*Art

[Adrian Wong]

Hello arth1,

For most users, it is true. You need less paging space with less memory. In fact, IF POSSIBLE, I personally would prefer to force Windows XP to use only system memory. There is no just substitute for fast system memory, if you want nothing but the best performance.

Now, there are a couple of reasons why I am sticking by my assertions that you do NOT have to increase your paging file size if you increase the amount of system memory.

1. Yes, a COMPLETE memory dump would require the paging file to expand to the size of the system memory + 1MB. But that's only if you truly require a complete memory dump.

Microsoft recommends sticking to a kernel memory dump, instead of a complete memory dump. Why? I'll quote them :-

For most purposes, a kernel memory dump file is the most useful kind of file for troubleshooting Stop messages. It contains more information than the small memory dump file and is significantly smaller than the complete memory dump file. It omits only those portions of memory that are unlikely to have been involved in the problem.

A kernel memory dump will require the paging file to be only about 1/3 of the system memory.

2. Even if you restrict your paging file to, for example, 500MB; Windows XP will automatically expand the paging file to store the memory dump BEFORE it is written out to disk on the next reboot.

Therefore, I consider it to be a real waste of hard disk space if you have 2GB of memory and yet create a 2GB paging file, just so Windows XP can write an enormous memory dump the next time it crashes.

However, I agree with your thoughts on the block size. The paging file uses a page size of 4KB, so it would be advisable to use a stripe size of 4KB for optimal performance.

[News Writer]

The article has just been updated!

Even today, virtual memory is still very important component of the operating system. No matter how much memory you have, there is always a need for virtual memory. Therefore, its performance is of great importance. Optimizing the virtual memory system will greatly improve the performance of the computer.

Today, we will take an in-depth look at data locality as well as memory dump files and their relationship with paging files. We will also take a look at the pros and cons of creating multiple paging files and moving the paging file to a RAID array. Come and check out the definitive Virtual Memory Optimization Guide!

Here are the updates:-

• Added a new section on dynamic paging files and data locality.
• Added a new section on huge paging files and memory dump files.
• Added a new section on creating multiple paging files.
• Added a new section on moving the paging file to a RAID array.

Link : Virtual Memory Optimization Guide Rev. 4.0!

[Justarius]

Excellent work on the article! Best I have read on the subject. I have a quick question that maybe I overlooked or isn't there.

At the end you mention that multiple drives will increase performance (as long as they are physical drives and not on the same IDE channel).

If I were to use multiple paging files on 2 drives, what size would I specify for each drive? Do I halve my "needs" between each drive or keep it the same? For example. Let's say I've calculated a 1 gig page file is best for my system and I have a permanent 1 gig page file on Drive 2 (Secondary Master). My OS being on Drive 1 (Primary Master). Now I've added a new hard drive on channel 3 (my motherboard has an onboard Promise controller) and I want to take advantage of multiple page files.

Do I make EACH of drive 2 and 3's page file 1 gig or should I make each 512MB? Keeping in mind I calculated that 1 gig is the optimal size.


Hopefully I made that clear. My question is if you want multiple page files on multiple drives, how do you allocate the size for each drive. Same as optimal, or 1/2 optimal (the only 2 that make sense to me)

Thanks for your time!

Justarius

[Choy]

Thanks for a very good and thorough guide.

There was one think I was wondering about, and that was the section about IDE channels. While it is true that only one device can be accessed at any one time, isn't it possible to access both devices sequencially before the first device has returned the data and for all intents and purposes get the double rate. I'm just speculating here, without any knowledge of how the IDE bus actually works.

[Adrian Wong]

Originally Posted by Justarius

Excellent work on the article! Best I have read on the subject. I have a quick question that maybe I overlooked or isn't there.

At the end you mention that multiple drives will increase performance (as long as they are physical drives and not on the same IDE channel).

If I were to use multiple paging files on 2 drives, what size would I specify for each drive? Do I halve my "needs" between each drive or keep it the same? For example. Let's say I've calculated a 1 gig page file is best for my system and I have a permanent 1 gig page file on Drive 2 (Secondary Master). My OS being on Drive 1 (Primary Master). Now I've added a new hard drive on channel 3 (my motherboard has an onboard Promise controller) and I want to take advantage of multiple page files.

Do I make EACH of drive 2 and 3's page file 1 gig or should I make each 512MB? Keeping in mind I calculated that 1 gig is the optimal size.


Hopefully I made that clear. My question is if you want multiple page files on multiple drives, how do you allocate the size for each drive. Same as optimal, or 1/2 optimal (the only 2 that make sense to me)

Thanks for your time!

Justarius

Justarius,

Many thanks for your kind comments.

As for your question, maybe I didn't make it clear enough. The size of the paging file should be calculated as a whole. That means no matter how many hard disks you want to split the paging file into, the amount of space dedicated to virtual memory should remain the same.

Therefore, if your optimal amount of virtual memory is 1GB, you should naturally split it into two 512MB paging files on two different hard disks (on separate IDE channels).

Hope that answers your question!

[Adrian Wong]

Originally Posted by Choy

Thanks for a very good and thorough guide.

There was one think I was wondering about, and that was the section about IDE channels. While it is true that only one device can be accessed at any one time, isn't it possible to access both devices sequencially before the first device has returned the data and for all intents and purposes get the double rate. I'm just speculating here, without any knowledge of how the IDE bus actually works.

Hello Choy,

The short answer is no. What you are possibly alluding to is command queuing and that feature is not available in most hard disks.

Even then, command queuing cannot fully compensate for concurrent accesses to the hard disk via separate IDE channels.

[coolmint]

Using diskeeper as an example, would the top or bottom part of the drive map represent the outer edge of the drive?

My pagefile is currently close to the bottom. I'm a little bit confused since the article said the outer edge was considered the beginning of the drive so I thought it would make sense that the outer edge would be at the top of the drive map.