» Welcome Guest
[ Log In :: Register ]

 

[ Track this topic :: Email this topic :: Print this topic ]

Topic: "SLOW" Bulk Tag Change for Some Files, Slow Bulk Tag Change< Next Oldest | Next Newest >
bothersome
Unregistered





Posted: May 14 2003,22:53

First off thanks for such a wonderful program.  Was in the middle of using it last week as I have been for what I believe to be about two years and decided that I finally had to register such a great program.  

Now my problem.  I'm an application support engineer so hopefully I've followed my own advise and have given you something good to go on.

Summary:  Some bulk tag changes seem to take forever.  The program freezes and it seems like it locks.  Sometimes this takes 10 minutes for one file and then the next file might have the same problem.

Notes:
1.  Does not happen when I use the custom file selection mode.
2.  Happens with many songs and is very frustrating
3.  Noticed only happens when I'm trying to set a date field (might be a red herring, but thought I'd mention anyways).
4.  If I successfully change the tag without custom file selection and then try a bulk tag change everything works fine.
5.  Has been happening for me with the last two releases as well and after registering.
6.  I've seen no pattern as to which files it happens on, but when it does I normally go get (and drink) a cup of coffee.

My system:
Windows XP SP1
512M PC2100 RAM
ATI Radeon 9000 64M DDR
nVidea Motherboard
sbLive 5.1

I'd be more than happy to put some debug code on my machine as I still have around 200 new CD's to go through (about 800 CD's currently in my collection => ) .  It's difficult to find which songs it's happening on and send those through to you as after the problem occurs the mp3 is OK.

Very grateful for your help.

Bothersome
Back to top
Disposable Hero
Old Board Geezer



Group: Super Moderators
Posts: 188
Joined: Aug. 2002
Posted: May 15 2003,02:35

You didn't mention which type of tags you're editing, though it may not apply. But I should note that ID3v2 tags are stored at the beginning of MP3 files. They often take quite a bit longer to write since the entire file sometimes has to be rewritten. This can be mitigated somewhat by adding 'padding' to the file, in case you need to modify or add tag data, but if the changes are greater than the padding a full rewrite will still be necessary. And Magnus will surely correct me if I have misrepresented anything ;)

Please do reply whether you feel that this is not the problem, or if it helps explain.
Back to top
Profile PM 
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: May 15 2003,12:58

Are the affected files extra large? At one time, on a specific computer, I noticed similar large pauses when processing files larger than 15 MB or so. I've never been able to reproduce it again though, not even on that same computer.

Also, if you perform an undo after processing some files, they should be completely restored to the prior state, every single byte of them, so you should be able to use that to get me a sample file to try it on. You can also use this to see if it happens to the same files if you do the same operation several times on your computer. If sending me such a sample file, please also include a description of the exact operation żou were doing to this file when it happened. Thanks.

PS. Thanks for being honest and registering.

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: May 15 2003,14:11

It's not with large files only and I modify both tags, but we're also not talking about just a long pause.  We're talking about the program locks up, doesn't display anything after I minimise lock up until it completes it's job.  Happens with about 1 CD out of 50 or so.

I'll try the undo thing next time it happens and see if I can get more information.

bothersome
Back to top
Disposable Hero
Old Board Geezer



Group: Super Moderators
Posts: 188
Joined: Aug. 2002
Posted: May 15 2003,19:50

Again, I don't know if this applies to your situation, but are the files stored on an external device? I've had wierd delayed-write lags on a USB2 drive; finally had to switch it to firewire.
Back to top
Profile PM 
TonyHughes
Unregistered





Posted: May 15 2003,22:10

I've had symptoms remarkably similar to 'bothersome's' throughout the 10 months I've been using your excellent program. During mass tagging as the program steps through the files it often hesitates, for a random period. It may be a second, ten seconds, or minutes. Fortunately the shorter times are more usual and longer times less usual. During this time CPU usage shows 100%. I've had several attempts to clarify what's happening, with little success. However a few points I've noticed are:-

1. The effect is during writing of tags only, preview mode and filename editing are not affected.
2. The effect is NOT repeatable. Doing the same operation again on the same folder does not duplicate the behaviour. So every attempt to try to identify particularly troublesome files has failed.
3. When a process is hesitating badly on a folder, closing TagStudio with ctrl-alt-del, then restarting again to perform exactly the same process on the same folder ALWAYS shows a dramatic improvement in speed over the whole folder.
4. The effect becomes more evident as the percentage of files with attached pictures increases. And there also seems to be a correlation with size of the pictures. (I realise it takes longer to write extra fields in this case. I have tested an extreme example by creating a folder of files all with 1MB picture. It then takes approx 1 second to modify each tag. However this is NOT the issue. The issue is a freezing for any time up to 45 minutes brfore stepping to the next file)

As I said I've tried some tests with inconclusive results in the past, here are 2 things I've tried today:

1. I chose an mp3 at random, 'removed all tag data', recreated tag, added a 300k picture to increase the likelihood of freezing.  Then I created 160 identical copies in a folder and used TagStudio to rename them 0001-0160.  Used 'autotag from filename' to write filename to TOFN.
After 5 minutes 2 files had been processed. ctrl-alt-del to stop program. Immediately reopened program and same operation ran smoothly through all 160 files in 43 seconds. Spent half an hour adding/removing/clearing tags in folder trying to repeat fault and everything ran perfectly.
The important thing here is that every file was identical, first time the first 2 froze program, remainder were untouched.  


2. I always add the filename to tag when backing up mp3s to cd. (so I can beat the system that can't store a 65 character filename in 700 000 000 bytes). With today's batch of 268 files from various sources I tried the undo test suggested above.
run 1:- 28 seconds. to tag from filename TOFN.
undo
run 2:- 28 seconds
undo
run 3:- 46 s
undo
run 4:- 1m 15s
undo
run 5:- 2m 10s  now files are freezing for long enough to see if the same ones are stopping each time.
run 6:- 2m 10s
undo
run 7:- 1m 45s many are pausing for a few seconds, I see one which has paused for 20s on 3 runs.
undo
run 8:- 1m 30s that same file didn't hesitate this time
undo
conclusion so far confirms all previous tests in that results never seem to make any sense.

I'm going to duplicate that suspect file. I now have 128 identical copies, apart from filename. in a new folder.
Running TOFN from filename this time has given an interesting result.
The randomly varying length of pause has gone.
Each file is processed either instantly, or within a time range of 14-18 seconds.  0, 1 or 2 are tagged instantly before the next freezes for 14-18 seconds. These are all identical files now. 15 minutes to write 128 tags.
There is 34MB free ram at this time, little else runnung.

I shall close TagStudio and repeat.
Just as I expected all 128 files were tagged in 13 seconds!

512MB RAM,  800Mhz processor, windows ME

Still love the program anyway.
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: May 16 2003,00:13

Thanks for the info and the tests Tony!

I don't have any explanation for why your test runs show such varying times (out-of-application factors are highly suspected for such a thing though, like operating system memory management and such), or for the complete lock-ups you said to experience with some files, but I have explanations for some of the things you mention though:


Tagging some files take 1-10 seconds, while others are instant:
When writing an ID3v2 tag to a file that does not already contain an ID3v2 tag (with enough padding), the entire contents of the file have to be rewritten (Disposable Hero mentions this above too). Depending on the size of the file, the speed of your storage device and the cache conditions of the storage device, this might take 1-10 seconds (and even more for really big files). Writing only ID3v1 tags, or writing ID3v2 tags to files that do already contain an ID3v2 tag (with enough padding) should always be practically instant though (with exceptions for situations related to my next explanation).


A bunch of files are tagged quickly, then it pauses on one file, then a bunch of files are tagged quickly again, then pauses, and so on:
This is due to the behavior of the so called "write caches" in modern storage devices. When you write data to a modern harddisk, it often stores the data in a fast internal cache memory chip. It then writes the data to the actual magnetic disk a moment later, when it is not busy serving requests from the user. This cache memory is only of limited size though, and several consecutive writes (of the type that e.g. a mass tagging operation in Mp3/Tag Studio produces) will fill up the memory completely. When this happens, data must be written to the magnetic disk of the harddrive before anything else can be done, and it also often takes the opportunity to dump all the data currently in the entire cache to the disk at this point. This can be several megabytes (often 4-20) of data, destined for scattered locations all over the disk, and writing it can hence take several seconds. Thus, it seems like the pause is only occurring for a certain single file, but it is actually the data of all the previous files (since the last "pause") that is written too. All this is done at a level way beyond the control of normal applications, sometimes even beoynd the control of the operating system. It is also very hard to reproduce the exact locations of these pauses when the same operation is repeated, since it is practically impossible to restore cache-conditions similar to any other time it was performed.


I hope this at least cleared some things up.

Also, I have used Mp3/Tag Studio quite much myself, on a pretty big collection, for a pretty long time (I promise, it's a registered copy ;)), and I have only experienced such a thing on one single instance out of all these numerous sessions. Hence, these problems might not all be entirely located in the program itself, but partially also on a lower level, out of my control (i.e. related to the local system of the user, like operating system, hardware and other factors). This hypothesis would also be supported by the fact that all such reported problems are very hard to reproduce, even on the same computer, and there is nothing in the code of Mp3/Tag Studio that will treat a certain file anything less than identical on any consecutive repetitions of the same operation.

But as always, if anyone has a sample file and/or operation that is able to reproduce any problem, you are very welcome to send them to me, and I will take a look!

--------------
Software author and website owner
Back to top
Profile PM WEB 
TonyHughes
Unregistered





Posted: May 16 2003,10:18

Thankyou for that detailed response. I used to think the freezing might be connected with the much greater tag size due to my added pictures, until I read that 'bothersome' has been experiencing it too. I can live with it for now, and I'll see if it was a quirk of my system when I eventually get a new machine.
Back to top
bothersome
Unregistered





Posted: May 18 2003,00:07

Two things...  one I'm not using an external drive.  two, the problem has happened on both my old machine and new machine with different hard drives in each.
Back to top
bothersome
Unregistered





Posted: May 18 2003,01:25

Found a folder that this is happening with.  It has 11 mp3's in it.  I've done more testing and found that...

1.  No matter what I choose to edit in the Bulk/Mass Edit the same thing happens.  100% CPU and very long times.
2.  Copying the folder and renaming doesn't change anything.
3.  Happens repetively after I undo on both copied folder and regular folder.

Next test is to try and eliminate files from the folder and see if "maybe" it's just one.  I've kept the original folder with the problem backed up.  Only problem is that it takes a "very" long time for me to get to the point where I can undo.
Back to top
bothersome
Unregistered





Posted: May 18 2003,01:29

3.  Problem also occurs if folder is copied to another physical drive (copied to C:\ same directory studio is installed on)
4.  Problem occurs on this directory but does not happen on all directories.  I can change to another folder, change things there, move back to folder and problem still happens...

Almost definitely is looking like something in the folder is causing studio to have a big hiccup...  you're talking about 8 minutes to change 11 tags (and studio is locked during this time and CPU is at 100%)
Back to top
bothersome
Unregistered





Posted: May 18 2003,02:18

OK...

1.  Problem does not happen with any 1 of the mp3's in the folder by itself.
2.  Problem does not occur on every combination of two mp3's but I have forwarded one such combination to you.

Hope it helps.  It's 0130 now ... quite tired, but really want to see if this problem can go away as I get it a lot more often than just "once".
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: May 18 2003,15:48

Thanks for your tests bothersome, it will be very interesting to see if I can reproduce this problem!

But I haven't received any email from you? I assume that the email you sent had a pretty big attachment, and thus were possibly dropped by a mail server between us. :( Sending emails with attachments larger than 2 MB per mail can sadly be quite unreliable. It would be great if you could either use something like WinRar to compress and split the archive up in 2 MB chunks, and send them all in separate emails, or contact me by email for further info about possible ways to transfer the files. Thanks!

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: May 19 2003,00:34

Wow... it's quite a bit bigger when divided into pieces.  I've just sent through 15 files about 450k each to your email, called 2.part<xx>.rar ... hopefully I did it all correctly as normally I don't create rar's just extract them...  ;)
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: May 19 2003,13:49

Thanks, I've got the files now. And even better, I was able to reproduce the program hanging! This will be interesting, I will hopefully be back with a report before the end of this week.

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: May 21 2003,10:28

Well, in my line of work (application support as I've said) it's always ...  "A reproduced problem is a fixed problem!!"
Back to top
bothersome
Unregistered





Posted: June 02 2003,12:17

Anything on this one yet?
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: June 02 2003,12:58

Sorry, haven't had time to analyze it yet, really swamped with work right now. :( It's still in the upper regions of my todo-list though. :) Hopefully soon.

--------------
Software author and website owner
Back to top
Profile PM WEB 
Nesta
Board Newbie



Group: Members
Posts: 8
Joined: Sep. 2002
Posted: June 03 2003,04:51

Fascinating problem and examination, guys. I'm actually on the edge of my seat reading about it! (I just picked up on the thread tonight.) And then the abrupt cliffhanger ending! I'll surely tune in next week, same bat-time, same bat-channel!! :-) I may even have experienced this temporary hang myself, if I recall correctly. But not in a great many moons, and not as often as you guys. And as evidenced by the sentiment of this thread, it certainly isn't a dealbreaker, not with as sweet a program as MP3 Tag Studio.

Nesta.
Back to top
Profile PM 
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: June 08 2003,15:28

I have tried to analyze this problem now, but it's very strange. I can only reproduce it randomly. The same operation only hangs sometimes and sometimes not (on the sample files from bothersome).

I have only gotten it to hang at one single time while being inside the debugger. When I then tried to see where the code was stuck, it seemed to be inside some Delphi memory management code, i.e. not my own code but code produced by the compiler.

Both these factors, the random nature of reproduction and the fact that the debugger could not map the code region where the execution was trapped to my own code, indicate that this problem might very well be due to a bug in the Delphi compiler or Delphi code library, and not in my own code.

Luckily, as reported by most users even having seen this problem once, it only seems to happen very rarely. So until I can get any kind of evidence that the problem might be in my own code, I guess we'll just have to live with it.

If anyone comes up with any more info, or more stable ways of reproduction, please post about it here and I will analyze it.

Thanks.

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: June 29 2003,17:18

Well, I have to say going through another 100 CD's today (after stopping last time after getting this problem), that after 5 CD's (folders) I already have this problem again.  This problem is definitely very irritating and happens more often than occasionally for me.

Any one else get it quite frequently?
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: June 29 2003,17:49

I'm sorry to hear about the trouble. :(
As I mentioned earlier in this thread, I once experienced that the program stalled as soon as I attempted to tag any file larger than 15 Mb or so. When I tried the exact same operation on the exact same files the next day (i.e. after the computer had been rebooted) everything worked like a charm, and it hasn't happened to me again ever since. Really gotta hate that kind of problems... :(

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: June 29 2003,17:54

15 times in the last half an hour!  Only been able to get through 10 folders!
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: June 29 2003,21:03

Have you tried rebooting?
And can you see any kind of connection between the files it hangs on? Are they e.g. extra big, like when it happened to me before?

--------------
Software author and website owner
Back to top
Profile PM WEB 
OrioN
Unregistered





Posted: July 30 2003,20:09

It's my turn!!

After a few years of nearly perfect use or success with the program, it now exhibiting the same behaviour as explained above. Both 3.01 & 3.05 are freezing up or stalling the program from any where from 1 to 15 minutes!!

Even WinXP Task Manager shows the program as "not responding" during this time.

What gives?

It just started misbehaving out of the blue!
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: July 30 2003,23:56

As mentioned above, anyone who can contribute with any clues or theories whatsoever as to which external conditions might trigger this behavior is very welcome to submit them here. :(

--------------
Software author and website owner
Back to top
Profile PM WEB 
OrioN
Unregistered





Posted: Aug. 01 2003,21:31

:shocked:

Dude!!

It's OrioN. I added a post the other day about the editor locking up or using all of a system's resources when batching a folder.

You replied and stated you needed help.

I may have found the issue!

On the folder that was consistently locking up, I tried to remove both the tags and frames, but yet again it locked up.

So I seached for a shareware editor to use in the interim and found Tag & Rename.

With Tag & Rename I noticed the albums folder's files contained art images in a tag field and I immediately removed them.

Guess what?  The files do not lock up your editor anymore.

I will try this on some more folders as they arise to confirm my findings...

PS. do you want a copy of the folder to test?

Check your email for me email address....

OrioN
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: Aug. 01 2003,21:55

Earlier tests have also pointed towards the fact that the problem often seems to have a higher probability of happening with files that have big tags. What sucks is that it seems very undeterministic, and does not behave the same with identical files on different computers, and hence I have not yet been able to reproduce it to a useful degree inside the debugger. :(

But I would gladly analyze your files anyway, and see if they produce any analyzeable problems on any of my systems. Just don't get your hopes up too soon, that's all I'm saying.

--------------
Software author and website owner
Back to top
Profile PM WEB 
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: Aug. 04 2003,02:37

YES!

Thanks to the sample files sent by OrioN, I finally managed to reproduce the problem inside the debugger.

The problem lies within the Delphi implementation of textfile-handling functions. They seem to use an inherently heap-trashing algorithm with quadratic time complexity, which leads to these indeterministic problems when working with extremely large rows (like the ones written to the undo-files when processing mp3:s with large ID3v2 tags, e.g. containing images).

I will have to change the format of the undo files to work around this. I have already made some successful tests, and it should be fixed in the next update.

Thanks everyone who contributed to solving this mystery, and especially OrioN who submitted the sample files that finally allowed me to reproduce the problem inside the debugger!

Stay tuned for the next update...

--------------
Software author and website owner
Back to top
Profile PM WEB 
LifeSquared
Unregistered





Posted: Nov. 01 2003,11:39

ahhh.... what a beautiful thing.  You are awesome magnus, the drama has been subsided for now.

i know this topic is old, but i've always enjoyed reading your wonderfully keen relations with your customers and their problems.  heck, even the non-customers (guilty... will be fixed soon though) get some help!

nesta STILL owes me a bag of popcorn that was taken during all the intense "seat-edge" flim-flammery
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: Nov. 01 2003,13:21

I'm glad you enjoy the support forum. :) I'm quite sad that I haven't been able to release the new update yet, but it's been SO much stuff going on with me moving and other things. Hopefully everything will calm down soon and the long awaited update will be released, it has some features that are worth waiting for anyway... :)

--------------
Software author and website owner
Back to top
Profile PM WEB 
drewsundin@hotmail.com
Unregistered





Posted: Nov. 22 2003,04:51

YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!YES!!

Was just about to go through another 200 albums (been putting it off since this bug....) I am so glad that this is reproduced and should be fixed soon!  You do not realize how much grief this has given me over the last year of using the software.  Thanks Orion for your work on this as well.  Thanks as always Magnus for such a great product.
Back to top
bothersome
Unregistered





Posted: Nov. 22 2003,04:54

PS... drewsundin@hotmail.com is bothersome.

Thanks again.
Back to top
bothersome
Unregistered





Posted: Dec. 31 2003,02:17

Not to sound sarcastic but has the new version turned into vaporware?  I check this site everyday for this new version and my backlog of mp3's to file has grown quite large....  just want to make sure that mp3 tag studio isn't the next Duke Nukem Forever...

3:-)
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: Dec. 31 2003,02:56

Living up to your name I see... ;)

I will really try to release a new update during January, but as usual I cannot promise anything. Why, oh why weren't we given more time in our lives... :ashamed:

--------------
Software author and website owner
Back to top
Profile PM WEB 
bothersome
Unregistered





Posted: Dec. 31 2003,16:41

If you have an alpha build of the new version that you need a hand with testing I have over 3000 mp3's to rename and tag.

Be more than glad to help test.

:)

My email address is drewsundin@hotmail.com or if it is too large then dsundin@airit.com

Thanks,

bothersome
Back to top
Magnus Brading
Almighty Author



Group: Super Administrators
Posts: 2751
Joined: Aug. 2002
Posted: Jan. 01 2004,23:59

I might have an alpha in a while, I have added you to my list of internal testers. Thanks.

--------------
Software author and website owner
Back to top
Profile PM WEB 
36 replies since May 14 2003,22:53 < Next Oldest | Next Newest >

[ Track this topic :: Email this topic :: Print this topic ]