Blacklist not working correctly

Requests for help go in here.
ChrisJ
Posts: 353
Joined: Sun Oct 27, 2013 3:32 am

Re: Blacklist not working correctly

Post by ChrisJ »

If I could get some clarity on the 'blacklist' file, on blacklisting an update manually and via AP... this would be much appreciated.

* A blacklisted file will be fully hidden from the drop-down list, even if ShowHidden=True? This is what I'm getting in 6137?
* If yes to the above, what is the quick way to remove 1 update from a list of 5 without deleting 'blacklist' outright - a manual delete in NP++?
* Is it possible to manually add files, following the same convention AP adds em - All Uppercase, exact name as apm minus extension?
* Is it possible to create a 'blacklist' from scrath via NP++, adding files manually, then saving the file with no extension in .\root directory as 'blacklist'?

The reason I ask is that with the releases I now have in .\modules AP is dreadfully long in loading, opening and closing to run experiments and do testing is getting to become a chore - about 3 minutes to fully load. I added /fast, some but not much in improvement. Sweeper' seems to work harder too!

This is what I start AP with - /fast /nolicense /verbose - if there's anything I can change that wont sacrifice a proper verification please suggest.

I might need to split releases up again.
User avatar
TheAPGuy
Site Admin
Site Admin
Posts: 979
Joined: Sun Oct 27, 2013 12:38 am
Location: California
Contact:

Re: Blacklist not working correctly

Post by TheAPGuy »

The blacklist is the first file to load when in "Install updates" mode. Then the modules are loaded. After each module is loaded AP checks that module against the blacklist. If it is in the list it is dumped rather then added to the list of modules already loaded. Manually adding modules via editing the file is possible but, difficult without using NP++. You can do it via looking for the UniqueID example... " UniqueID=WINDOWS_COMMON_PARENT " Copy paste out the "WINDOWS_COMMON_PARENT" into the blacklist and you just killed the entire Common branch... giving errors with the lower leafs of course since they will no longer have a parent branch to cling too (ahh the wonderful fall leafs).

Are you testing with an AIO setup? Using the settings you are using are the fastest. There is room for me to increase module loading by loading the file into memory first before dissecting it but, not much to improve the verification process.
Using /fast skips both size and MD5 hashing. It only checks if the file name exists. That is a very fast check however, if you are using an AIO there are thousands of files to check. Windows 7 x64 has over 500 by itself. If you want to do testing... I recommend creating a test script like I do. Remove all update install files. This should cut your load times in half. The apms will still load and show you the install option. The only downside is that AP will flag as unofficial.

AP does not care if the entry to the blacklist is uppercase or lower case. It just adds them in as uppercase because its been our convention to put the UID in with all caps and copies that entry into the blacklist.
* If yes to the above, what is the quick way to remove 1 update from a list of 5 without deleting 'blacklist' outright - a manual delete in NP++?
You mean you blacklisted 5 files and in there you want to remove one from the list? Yes, you have to edit the file and remove the entry. Blacklisting anything is supposed to be permanent after all. (at least from my point of view)

Good?
ChrisJ
Posts: 353
Joined: Sun Oct 27, 2013 3:32 am

Re: Blacklist not working correctly

Post by ChrisJ »

TheAPGuy wrote:UniqueID=WINDOWS_COMMON_PARENT " Copy paste out the "WINDOWS_COMMON_PARENT" into the blacklist and you just killed the entire Common branch... giving errors with the lower leafs of course since they will no longer have a parent branch to cling too (ahh the wonderful fall leafs).
Thanks for the info, that was helpful.

I have no need to hide a whole category but it's nice to know you can. I suppose I would add the parent and the lower leafs to the list to avoid a messy log. I'll assume there's no limit to the size of the blacklist file (in KB, MB) or the length of the list (# of files) :?: Most of my reason to employ the list is to remove (once installed) from AP the long titled updates that force a horizontal scrollbar, don't know why but that annoys me. I would make use of the "custom" folder you mentioned a while back but it never worked regardless of what I tried, flagged AP as unofficial.

What I hope to do is find the rti tool again, maybe edit the apm's 'title' to a uniform length (most are fine), move important info to 'description'. Hopefully this will allow me to load AP at a size I prefer and remove the scrolling.
TheAPGuy wrote:...Are you testing with an AIO setup?
... if you are using an AIO there are thousands of files to check.
...If you want to do testing... I recommend creating a test script like I do. Remove all update install files. This should cut your load times in half. The apms will still load and show you the install option. The only downside is that AP will flag as unofficial.
...Yes, one .\modules folder. AP is too slow now, and removing verification is not a reasonable fix IMO. I'll change this soon, remove '/fast', and create multiple instances of AP seems an OK compromise - IMO, for me anyway. Dropping XP, 7, and 8 (x64, x86) into their respective folders, .NET (x64, x86) into their own, etc, this is doable.

* If /fast skips both size & md5 you might want to update /help - I used /fast because it was only supposed to bypass md5, not size. I figured as long as size hadn't changed md5 shouldn't have either - assuming size is calculated down to the byte.

...Exactly, I haven't downloaded .NET (x64, x86), 7 or 8 (x86) yet... 3.8GB, 2100+ files seems too much in one release as it stands now. I wonder how long it takes AP to load and verify say 10+GBs of data - YIKES!

...I'm not exactly sure what you're testing here, the apm only..? 'the install option'..?
TheAPGuy wrote:AP does not care if the entry to the blacklist is uppercase or lower case. It just adds them in as uppercase because its been our convention to put the UID in with all caps and copies that entry into the blacklist.
OK.
TheAPGuy wrote:
...what is the quick way to remove 1 update from a list of 5 without deleting 'blacklist' outright - a manual delete in NP++?
You mean you blacklisted 5 files and in there you want to remove one from the list? Yes, you have to edit the file and remove the entry. Blacklisting anything is supposed to be permanent after all. (at least from my point of view)

Good?
Yes, remove 1 file from the blacklist. I figured since AP created the list, added files, I wondered if files might be removed via AP too. NP++ is fine.

I'm good :mrgreen:
User avatar
TheAPGuy
Site Admin
Site Admin
Posts: 979
Joined: Sun Oct 27, 2013 12:38 am
Location: California
Contact:

Re: Blacklist not working correctly

Post by TheAPGuy »

ohh damn, I forgot I commented the size code out. For a while I was thinking of having /fast do a size check instead of skipping everything. I didn't put it back in. ohh well, next version it will switch to size checking on /fast.

btw, yes size is to the byte.

ohh a warning to adding a ton of entries into blacklist... the more you add the longer the module load time will be. It has to do an entry check against the entire list each time a module loads. While this is fairly fast it adds up when you have thousands of entries. I have not much of idea how much time it would add though... 10 seconds?
ChrisJ
Posts: 353
Joined: Sun Oct 27, 2013 3:32 am

Re: Blacklist not working correctly

Post by ChrisJ »

TheAPGuy wrote:<snip...>
ohh a warning to adding a ton of entries into blacklist...
Thanks for the heads-up on adding too many files to the blacklist, I don't think I'd add more than a handful anyway.
older-than-dirt
Posts: 55
Joined: Sun Jan 05, 2014 10:51 pm

Re: Blacklist not working correctly

Post by older-than-dirt »

Thanks for looking into this.
I'm probably going to wait for the changes in detection stuff in .36->.37->.38 to settle down before testing this. I'll start using .32 with the knowledge that the blacklist stuff isn't working, but I've lived without it for the past 7 years or so, so I don't think waiting a little bit longer is going to bother me too much.

It looks like I have to move on from APUP. It now fails (during the download of some file) for both the Win 7 and Win 8 updates with the latest (13-OCT-15) lists, but running 6.1.32 succeeds with both of them so I'm assuming that APUP is no longer supported and/or tested. Just as well...
ChrisJ
Posts: 353
Joined: Sun Oct 27, 2013 3:32 am

Re: Blacklist not working correctly

Post by ChrisJ »

older-than-dirt wrote:Thanks for looking into this.

...I'm probably going to wait for the changes in detection stuff in .36->.37->.38 to settle down before testing this. I'll start using .32 with the knowledge that the blacklist stuff isn't working, but I've lived without it for the past 7 years or so, so I don't think waiting a little bit longer is going to bother me too much.

...It looks like I have to move on from APUP. It now fails (during the download of some file) for both the Win 7 and Win 8 updates with the latest (13-OCT-15) lists, but running 6.1.32 succeeds with both of them so I'm assuming that APUP is no longer supported and/or tested. Just as well...
...Since there's not much of a changelog I can't speak too specifically but you might be missing out on other improvements in later editions of AP, your call :)

...Always provide a log file as a first step - run APUP or AP on the supposed failing script and provide an unedited log file, otherwise it's hard to know if APUP is having issues in a particular environment or if the script is the issue. It appears for you it might be the environment APUP is running in. I ran APUP tests a few weeks back and APUP seemed to work fine overall even on a school computer. There were a few issues but I think it might have been at my end when using my laptop, APUP running on the school PC worked perfectly.
User avatar
TheAPGuy
Site Admin
Site Admin
Posts: 979
Joined: Sun Oct 27, 2013 12:38 am
Location: California
Contact:

Re: Blacklist not working correctly

Post by TheAPGuy »

He is probably running into HTTPS errors. Apup doesn't support HTTPS and WAC's newer scripts are using them since AP does support HTTPS. It leaves less editing for WAC and some sources are HTTPS only. Some admin script items for example.
Pendraguin
Posts: 97
Joined: Mon Oct 19, 2015 5:52 am

Re: Blacklist not working correctly

Post by Pendraguin »

The blacklist did not work for me in 6.1.32 but it seems to be working fine in 6.1.37 and 6.1.38.
User avatar
Whatacrock
Release Maintainer
Release Maintainer
Posts: 1967
Joined: Mon Oct 28, 2013 10:47 am
Location: Australia
Contact:

Re: Blacklist not working correctly

Post by Whatacrock »

@older-than-dirt --- to download the latest scripts you will 6.1.32 or greater to download the releases, Microsoft has started using HTTPS for their downloads
Just thought I would add my 1.5 cents worth in....
"Now if you Sons of B*@ches got anything else to say, NOW'S THE F@#%ING TIME!!"
Post Reply