Insimenator.org
March 28, 2024, 10:55:53 am *
Welcome, Guest. Please login or register.

Login with username, password and session length
News:
 
  Home   Forum   Help Search Calendar Login Register  
Pages: [1]
  Print  
Author Topic: A New Wrinkle in Package Dependencies and Load Ordering  (Read 5844 times)
0 Members and 1 Chinese Bot are viewing this topic.
jase439
Member

Posts: 1172


View Profile
« on: April 08, 2006, 08:23:09 pm »

It has been a long held belief that files in the downloads folder are opened in alphabetical order, folders coming before files in the same folder. Once a resource is loaded once, it's not loaded again.

Indeed this is something that has long been accepted as truth (indeed I was one of the first people asserting this observation as fact), and most of the modding community now accepts the "alphabetical load ordering" theory as fact.  However, something recently surfaced during the InTeenimater OFB beta that made me dig a little bit deeper into this and now leads me to believe this is not universally true and is potentially the source of many headaches for many people who may have hundreds - even thousands - of mods in their game.

After careful analysis using a number of file IO tools, I discovered that the game loads package files in the same sequence as the underlying file system sees them.  On Windows XP or Windows 2000 running an NTFS file system, files are always indexed alphabetically with subfolders appearing after (not before).  In essence it is "breadth-first" traversal of the directory structure and then alphabetical by file.  Indeed this is exactly the same order that we have all come to expect with respect to package load ordering.  I would say, that this holds for the large majority of Sims 2 players (fortunately!)

However, one of my beta testers experienced a phenomenon that nobody else on the test team could replicate: their InTeenimater flavor paks were being loaded BEFORE the core InTeenimater packages (flavor paks are add-ons/extensions to InTeen that are designed to augment the base feature set and intended to load after the core package files).  This is despite the fact that the flavor paks follow alphabetically AFTER the core packages.  Naturally, I assumed this would be sufficient.  Indeed many authors assume this, as there are a whole host of mods that start with 'z' in their name to ensure late loading.

I wrote a little program (attached), which enumerates the directory structure in the same order that the game "sees" the files.  What I observed may surprise you:

With the exception of the one beta tester I mention, everyone (including myself) experienced identical alphabetical ordering:

ME_MindControlMirror.package
InSIMenator (UV) v2.3 DEST Edition.package
InTeenimater_A.package
InTeenimater_B.package
InTeenimater_C.package
InTeenimater_D.package
InTeenimater_E.package
InTeenimater_F.package
InTeenimater_FlavorPak_BackToSchool.package
InTeenimater_FlavorPak_CollegeAdmissions.package
InTeenimater_FlavorPak_NoAgeOfConsent.package
InTeenimater_FlavorPak_NoCommittedRelationships.package
InTeenimater_FlavorPak_NoFailBirthControl.package
InTeenimater_FlavorPak_NoMiscarriage.package
InTeenimater_FlavorPak_ResidentialGraduates.package
InTeenimater_FlavorPak_SameSexPregnancy.package

Not surprisingly, each one of these testers have an NTFS file system running on Windows XP.  NTFS is the default file system for all new Windows XP installations.

However, the one beta tester I mentioned, running a FAT32 file system, experienced this ordering:

InSIMenator (UV) v2.3 DEST Edition.package
ME_MindControlMirror.package
InTeenimater_FlavorPak_SameSexPregnancy.package
InTeenimater_FlavorPak_BackToSchool.package
InTeenimater_FlavorPak_CollegeAdmissions.package
InTeenimater_FlavorPak_NoAdultTeens.package
InTeenimater_FlavorPak_NoAgeOfConsent.package
InTeenimater_FlavorPak_NoCommittedRelationships.package
InTeenimater_FlavorPak_NoFailBirthControl.package
InTeenimater_FlavorPak_NoMiscarriage.package
InTeenimater_FlavorPak_ResidentialGraduates.package
InTeenimater_F.package
InTeenimater_A.package
InTeenimater_B.package
InTeenimater_C.package
InTeenimater_D.package
InTeenimater_E.package

Indeed, this is the EXACT sequence in which the files were physically written to the hard disk.  As you can see, the files are "almost" alphabetical within their respective groups (the core InTeenimater files are mostly sorted, for instance, with the exception of the F package)...but the flavor paks PRECEDE the core InTeenimater files.  As it turns out, InTeenimater_F happens to be the first file in the ZIP file that I distributed to the beta testers, hence the reason it appears first in this list as well.  This user installed Merola's Mind Control mirror AFTER the InSIMenator, placing it 2nd instead of first in this listing.

Only FAT32 systems exhibit this phenomenon.  For my InTeenimater users this is not likely a huge issue, since most people will install the flavor paks AFTER the core program.  But if someone unpacks them in the opposite order, they will have HUGE problems (jump bugs, crashes, missing menu options, etc.)

As an additional test, I had this person delete the flavor paks and then recopy them back into their Downloads folder and run the utility.  This is the order that those files then appeared:

InSIMenator (UV) v2.3 DEST Edition.package
ME_MindControlMirror.package
InTeenimater_F.package
InTeenimater_A.package
InTeenimater_B.package
InTeenimater_C.package
InTeenimater_D.package
InTeenimater_E.package
InTeenimater_FlavorPak_BackToSchool.package
InTeenimater_FlavorPak_CollegeAdmissions.package
InTeenimater_FlavorPak_NoAgeOfConsent.package
InTeenimater_FlavorPak_NoCommittedRelationships.package
InTeenimater_FlavorPak_NoFailBirthControl.package
InTeenimater_FlavorPak_NoMiscarriage.package
InTeenimater_FlavorPak_ResidentialGraduates.package
InTeenimater_FlavorPak_SameSexPregnancy.package

In this configuration, the user no longer experienced crashes or jump bugs, and it was evident that the InTeenimater was functioning normally again.

Placing the flavor paks in a subfolder also proved to be an effective means of ensuring that they loaded after the files in the root, again suggesting that sub folders are processed *after* the files in the current directory.

This news throws an obvious wrinkle into the equation for those of us with inter-dependent mods that require correct load ordering in order to function properly.  We should have a care that those users who are running Win98/ME and those upgrading from 98 to XP, are likely to have FAT32 file systems and that we cannot rely on alphabetical package naming to ensure proper load ordering on these systems!

J
« Last Edit: April 08, 2006, 08:56:21 pm by jase439 » Logged

"Craaaazy...toys in the attic...I am craaaazy...truly gone fishing...they must have taken my marbles away..."
BeosBoxBoy
Silent

Gender: Male
Posts: 5021



View Profile WWW
« Reply #1 on: April 09, 2006, 12:23:22 am »

Jase, in all honesty, the file creation sequence has always been the read sequence on my FAT32 partition - I favour it over NTFS for non-game reasons.

Here is also another bit of info for you.  The only items that appear out of the sequence you give are those files I receive from Marvine, this I attribute to the charset difference between her French Windows XP and my Cyrillic Windows XP.

In my experience all custom content I create with Latin charset names appear first followed in descending order by file creation date (as per first write to hard drive) - then all things by Marvine begin.

So I have never "enjoyed" alphabetical sequencing.

EDIT:  As a work around, I have found that dragging the files from the RAR archive avoids this charset issue and restores the write-date ordering; possibly due to a charset limitation in the English version of WinRAR software I use.
« Last Edit: April 09, 2006, 12:27:00 am by ~Marvine~ » Logged

"There is a certain elegance in wasting time. Any fool can waste money, but when you waste time you waste what is priceless."
-- Maugham, W. Somerset. Ashenden: Or the British Agent.
jase439
Member

Posts: 1172


View Profile
« Reply #2 on: April 09, 2006, 01:39:04 am »

Quote from: beosboxboy
In my experience all custom content I create with Latin charset names appear first followed in descending order by file creation date (as per first write to hard drive) - then all things by Marvine begin.
Character sets are yet another wrinkle that is almost impossible to account for *sigh*.  I really wish Maxis had provided a deterministic means of package loading.  Some elements can be versioned (such as BHAV's), but in all my testing, the version field is ignored for the purposes of package loading rendering it pretty much useless.

The purpose of this post is to raise awareness that long-held belief that the game loads package files in an alphabetical "breadth-first" manner is something enjoyed only by those with an NTFS file system.  The age-old practice of "slapping a 'z'" to the front of a file name in order to delay loading to the last possible moment is not as ironclad as once thought.

Anyway, here is the utility I mentioned that dumps the package folders.  It should dump the packages in the same order as they are loaded by the game.  I've provided a unicode and non-unicode release for Win9x/ME users.

* modlist.zip (25.32 KB - downloaded 340 times.)
* modlist98.zip (24.24 KB - downloaded 340 times.)
« Last Edit: August 09, 2006, 11:15:08 pm by jase439 » Logged

"Craaaazy...toys in the attic...I am craaaazy...truly gone fishing...they must have taken my marbles away..."
Pages: [1]
  Print  
 
Jump to:  


Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.038 seconds with 31 queries.
SimplePortal 2.1.1