Virtual Memory Optimization Guide!

Discussion in 'Reviews & Articles' started by Dashken, Oct 29, 2004.

  1. Scarceas

    Scarceas Newbie

    I think I read it in the forums at Storagereview.com, but it's been ages ago and I don't have a link to the particular post.

    It was an interesting enough idea that it stuck in the back of my mind, and when I was reading your guide I remembered it, and wondered if it was important or not... Guess not? ;)
     
  2. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Well, the trouble with statements made by people in the forums is that anyone can say anything. :mrgreen:

    That's the reason why we prefer to write and post articles. This allows people to critique. We can then refine it.

    In the end, we hope to present definitive conclusions on the particular topic, instead of thousands of "opinions" on the same topic.
     
  3. arth1

    arth1 Newbie

    Memory size

    The article repeating a common error in stating that on modern systems with lots of ram, you need less paging space. If you're a developer, or have production systems with support, you almost certainly want to be able to save a paging file dump if the system crashes. That requires that your paging file is large enough to hold a system memory dump. Due to what's similar to a file header in a swap file, that means you need at least 1xRAM+1 block of swap.
    For users who neither develop nor receive support on their system which might find a full dump useful, they obviously don't have this requirement. But many of us do, and it's a disservice to ignore this.

    As for using multiple paging files on multiple drives, Windows NT/2000 will automatically stripe when using two or more paging files (or rather, always write a page to the lowered numbered paging file with the most space free, which amounts to striping in practice). Just set the max size to the same size, and you'll get a striping effect. For XP, the same holds true, but if one drive is busy, the next drive will be immediately tried, meaning for single-block pages, XP is often faster, but for large pages, it evens out.

    Finally, make sure that the block size on the drive is equal to or a multiple of the ram page size. Unless running weird hardware (like NT 3.5 on RISC), the RAM page size will normally be 4k, so choosing a 4k block size while formatting the drive will be ideal, while 8k/16k/32k/64k also works, but won't make for optimal "striping" if using multiple paging files on multiple drives.

    Regards,
    --
    *Art
     
    Last edited: Nov 9, 2004
  4. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Hello arth1,

    For most users, it is true. You need less paging space with less memory. In fact, IF POSSIBLE, I personally would prefer to force Windows XP to use only system memory. There is no just substitute for fast system memory, if you want nothing but the best performance.

    Now, there are a couple of reasons why I am sticking by my assertions that you do NOT have to increase your paging file size if you increase the amount of system memory.

    1. Yes, a COMPLETE memory dump would require the paging file to expand to the size of the system memory + 1MB. But that's only if you truly require a complete memory dump.

    Microsoft recommends sticking to a kernel memory dump, instead of a complete memory dump. Why? I'll quote them :-

    A kernel memory dump will require the paging file to be only about 1/3 of the system memory.

    2. Even if you restrict your paging file to, for example, 500MB; Windows XP will automatically expand the paging file to store the memory dump BEFORE it is written out to disk on the next reboot.

    Therefore, I consider it to be a real waste of hard disk space if you have 2GB of memory and yet create a 2GB paging file, just so Windows XP can write an enormous memory dump the next time it crashes.

    However, I agree with your thoughts on the block size. The paging file uses a page size of 4KB, so it would be advisable to use a stripe size of 4KB for optimal performance.
     
  5. News Writer

    News Writer News Writer

    The article has just been updated! :wave:

    Even today, virtual memory is still very important component of the operating system. No matter how much memory you have, there is always a need for virtual memory. Therefore, its performance is of great importance. Optimizing the virtual memory system will greatly improve the performance of the computer.

    Today, we will take an in-depth look at data locality as well as memory dump files and their relationship with paging files. We will also take a look at the pros and cons of creating multiple paging files and moving the paging file to a RAID array. Come and check out the definitive Virtual Memory Optimization Guide!

    Here are the updates:-
    • Added a new section on dynamic paging files and data locality.
    • Added a new section on huge paging files and memory dump files.
    • Added a new section on creating multiple paging files.
    • Added a new section on moving the paging file to a RAID array.

    [​IMG]

    Link : Virtual Memory Optimization Guide Rev. 4.0!
     
  6. Justarius

    Justarius Newbie

    Quick question on sizing with multiple drives

    Excellent work on the article! Best I have read on the subject. I have a quick question that maybe I overlooked or isn't there.

    At the end you mention that multiple drives will increase performance (as long as they are physical drives and not on the same IDE channel).

    If I were to use multiple paging files on 2 drives, what size would I specify for each drive? Do I halve my "needs" between each drive or keep it the same? For example. Let's say I've calculated a 1 gig page file is best for my system and I have a permanent 1 gig page file on Drive 2 (Secondary Master). My OS being on Drive 1 (Primary Master). Now I've added a new hard drive on channel 3 (my motherboard has an onboard Promise controller) and I want to take advantage of multiple page files.

    Do I make EACH of drive 2 and 3's page file 1 gig or should I make each 512MB? Keeping in mind I calculated that 1 gig is the optimal size.


    Hopefully I made that clear. My question is if you want multiple page files on multiple drives, how do you allocate the size for each drive. Same as optimal, or 1/2 optimal (the only 2 that make sense to me)

    Thanks for your time!

    Justarius
     
  7. Choy

    Choy Newbie

    Thanks for a very good and thorough guide.

    There was one think I was wondering about, and that was the section about IDE channels. While it is true that only one device can be accessed at any one time, isn't it possible to access both devices sequencially before the first device has returned the data and for all intents and purposes get the double rate. I'm just speculating here, without any knowledge of how the IDE bus actually works.
     
  8. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Justarius,

    Many thanks for your kind comments. :)

    As for your question, maybe I didn't make it clear enough. The size of the paging file should be calculated as a whole. That means no matter how many hard disks you want to split the paging file into, the amount of space dedicated to virtual memory should remain the same.

    Therefore, if your optimal amount of virtual memory is 1GB, you should naturally split it into two 512MB paging files on two different hard disks (on separate IDE channels).

    Hope that answers your question! :mrgreen:
     
  9. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Hello Choy,

    The short answer is no. What you are possibly alluding to is command queuing and that feature is not available in most hard disks.

    Even then, command queuing cannot fully compensate for concurrent accesses to the hard disk via separate IDE channels.
     
  10. coolmint

    coolmint Newbie

    Using diskeeper as an example, would the top or bottom part of the drive map represent the outer edge of the drive?

    My pagefile is currently close to the bottom. I'm a little bit confused since the article said the outer edge was considered the beginning of the drive so I thought it would make sense that the outer edge would be at the top of the drive map.
     
  11. Justarius

    Justarius Newbie

    Hey Adrian, thanks for the response. After reading your VM guide I went out and bought your BIOS book (your guides make excellent advertisements for your book just by their quality). I have one more quick question for the VM gurus!

    I'm still a little confused on what will be optimal page settings based on my setup. I have 2 x 160GB ATA100 drives that are Raid 0 via a promise controller. I also have 2 more large ATA100 drives (250GB and 120GB) on the Primary and Secondary controllers (same speeds but since not RAID, slower than the 2x160s of course). Where to put the Page File vs the OS/Apps?

    I would think to put XP and programs on the Raid 0 drives as they are fastest... my HDDs access Windows components and programs more often than paging right? Bottom line is to have Adobe Premiere and Half-Life 2 running the best. ;) So then where to put the page file...

    I can stick it 1/2 on the OS/Apps/Games RAID 0 drives and 1/2 on another. Or all on another. Or split between the 2 others. Or all on the OS/Apps drive. Or if it makes THAT much diff, put the OS/Apps on a different drive and use the RAID drives for Page File plus storage...

    Any quick advice? From the guide I understand A) Multiple drives is better. B) RAID 0 is better but what if you have both? ;)

    Right now I'm leaning towards OS/Apps on RAID 0 drives. Page file split between drives 3 and 4 (although drive 4 is shared with a DVD-ROM/CD-RW drive). Pros - OS/Apps on fastest drive. Page file not shared with OS/Apps. Cons - Page file on drives slower than OS/Apps. Page file on drives routinely accessed for storing and moving large data files (and occasionally shared with burner).

    Help! I'm drowning in the options... :)

    Justarius
     
  12. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Hello coolmint,

    The top of the drive map always represents the outer tracks of the hard disk. :mrgreen:
     
  13. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Oh, wow.. Thanks for your support! :thumb:

    Heh.. When it comes to such complicated setups, the best option is never quite clear-cut. :mrgreen:

    But I agree with your thinking. It should be better to put your OS and applications on the RAID array. Data, if possible, should be kept on the single drives, to reduce the possibility of catastrophic data loss when anything happens to the RAID array.

    The paging file would definitely benefit from the RAID 0 array. And the higher risk of data loss in a RAID 0 array won't matter to the paging file. But splitting the paging files between the two single drives would essentially give you the same effect.

    Still, I think you should consider splitting the paging file into three drives. One on each single drive and one on the RAID 0 array.

    Windows XP will automatically use the fastest (at that moment) paging file for virtual memory. So, if it's busy using the RAID 0 array, it will skip the paging file on the RAID 0 array for the paging files on the single drives.

    Hope that helps you some! :mrgreen:
     
  14. News Writer

    News Writer News Writer

    The article has just been updated! :wave:

    Even today, virtual memory is still very important component of the operating system. No matter how much memory you have, there is always a need for virtual memory. Therefore, its performance is of great importance. Optimizing the virtual memory system will greatly improve the performance of the computer.

    Today, we will take an in-depth look at moving the paging file to a RAM drive. We will also take a look at how to reduce reliance on the paging file to improve the performance of the system. Come and check out the definitive Virtual Memory Optimization Guide!

    Here are the updates:-
    • Moving the paging file to a RAM drive
    • Reducing reliance on the paging file

    [​IMG]

    Link : Virtual Memory Optimization Guide Rev. 4.0!
     
  15. Europa2010AD

    Europa2010AD Newbie

    I am just wondering, if I put my paging file in the very last partition of my harddrive (only one physical harddrive), does that mean it is located in the outer or inner tracks of the harddisk?

    I am aware that you mentioned how it is unnecessary to put the paging file in a separate partition in the article, but I still did it because it would save me a few CDs when I make backup images of my primary partition using programs like Norton Ghost.
     
  16. Chai

    Chai Administrator Staff Member

    The last partition means it's inside the platter which is of course the slowest.
     
  17. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Yeah, as mentioned in the guide, the first partition starts with the outer tracks and subsequent partitions are further in and are therefore slower.

    The last partition is always the slowest partition.

    Hope that clears it up for you! :mrgreen:
     
  18. brainstormer

    brainstormer Newbie

    NTFS ans FAT32

    Hi.

    After reading this article I was wondering something because my Windows XP is running on a partition with NTFS but my paging file is on a drive with FAT32. Is this bad, or do you think that doesn´t matter?

    //Klaus
    Denmark
     
  19. Adrian Wong

    Adrian Wong Da Boss Staff Member

    Hello brainstormer,

    That's certainly food for thought. :think:

    FAT32 is faster than NTFS. And the paging file does not need the improved data integrity offered by NTFS. So, yes, putting the paging file on a FAT32 partition would be better.

    However, I think this is a minor benefit. Certainly nothing compared to moving it to the outer tracks or defragmenting it. :think:
     
  20. Hello again and thank you for the great addition to the guide!

    My current setup in the last few months is:

    Drive A- WD Raptor 36
    Drive B- WD 120GB 8MB cache
    both SATA on different channels ofcourse.,

    Drive A 200MB page start of the drive NTFS
    Drive B 500MB page in the first 1GB partition FAT32

    I'm not into "is it good?" kind of questions., I am just looking for some critique and advices., from anyone can help,.
    My computer is running in good performance but as everyone I'm looking for the best ;)
    needles to say that by my testings I got maximum of 600MB page (by the task manager>performance tab>.

    Thank you very much!
    -Noam
     

Share This Page