deleting double words in a text file (WXP-Pro)

[mattmac24]

hey, i have a very long text file with lots of words in it, each word is on a new line.

i want a script that will delete a word if it appears twice in the file so it only appears once.

i woughfly have an idea of how to do it but need a little assistanse putting it togeather.


here is my idea:

read the file (c:\text.txt) (can do sepretly but dont know how to join step 1 and 2)
slpit each line using the split function into an array(can do)
check each line against the other lines(no idea)
delete the double or triple line(know how to delete but not to match up an array with a text file line)


i think that might work but need some help.
your help would be gratly appreciated.


Matt

[cprasley]

If the text file is already one word per line then you don't need to split the input.

In any case, you should use a Dictionary Object. As you read each word you just add it to the dictionary as a key (with an item value of "true"). Duplicates would be ignored. To retrieve the word list, extract the keys using the VBArray type, sort the array and then iterate.

You didn't say if you were using JS or VBS. Here's a quick JS example:

var WordDict = new ActiveXObject("Scripting.Dictionary");
FH = FileSystem.OpenTextFile("input.txt");
while(!FH.AtEndOfStream) WordDict.Item(FH.ReadLine().toUpperCase()) = true;
FH.close();
WordArray = new VBArray(WordDict.Keys()).toArray().sort();
FH = FileSystem.CreateTextFile("output.txt",true);
for(index in WordArray) FH.writeLine(WordArray);
FH.close();

The dictionary keys are case-sensitive, so you'll probably want to fold them all up or down (as I did) to catch all the dups.


<P ID="edit"><FONT class="small">[i]Edited by cprasley on 04/03/04 15:01.</FONT></P>

[mattmac24]

i ment a VBS file but it dont really matter.

just that i dont no anything about js(javascrip?)

i renamed the file test.js and now dreamweaver keeps opening it up so it wont even run.

som1 please help lol...im so bad at this


thank for doin that for me now i just need som1 to tell me how to use it

thanks again matt

[meagain35]

I would recommend you either change your settings in dreamweaver to exlude .js files. or modify your registry:
Here is the export:
Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\JSFile\Shell\Open\Command]
@=hex(2):25,00,53,00,79,00,73,00,74,00,65,00,6d,00 ,52,00,6f,00,6f,00,74,00,25,\
00,5c,00,53,00,79,00,73,00,74,00,65,00,6d,00,33,00 ,32,00,5c,00,57,00,53,00,\
63,00,72,00,69,00,70,00,74,00,2e,00,65,00,78,00,65 ,00,20,00,22,00,25,00,31,\
00,22,00,20,00,25,00,2a,00,00,00

or you can manually edit it to open with wscript.exe:
HKEY_CLASSES_ROOT\JSFile\Shell\Open\Command
Default="%SystemRoot%\System32\WScript.exe "%1" %*"

[mattmac24]

ok well ive never used a .js file before and am completly hopeles at this...i know you just wanted to show me how its done but can som1 plz just spare a few mins to complete it?

coz im soo lost,

thx, matt

[cprasley]

You can launch test.js from the command prompt using "CSCRIPT TEST.JS"

Note that my sample script opens the file "input.txt" and creates "output.txt" with the sorted list of unique words. If your word-list is still text.txt then you'll need to rename it or edit the script.