Wikipedia:Reference desk/Archives/Computing/2014 January 31

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

January 31

Other sites like Project_Euler

Besides the site above, what other well edited sites have interesting math problems meant to be solved with programming (but no language is specified). OsmanRF34 (talk) 13:45, 31 January 2014 (UTC)

Try Googling "sites like Project Euler" and you should be able to find some others, there are some blog posts in the results which are lists of similar sites. --Canley (talk) 09:23, 1 February 2014 (UTC)

That's easier said than done. Specially if I insist in the well-edited part, but also the math with computing is rare, unless puzzles are a part of math (they are somehow, but not quite) and they are not all language independent either. Many don't have the problems ordered by difficulty. OsmanRF34 (talk) 12:59, 1 February 2014 (UTC)

I recall AMS used to publish monthly brainteasers in their student newsletter. You can sign up for their newsletter by email, and find links to several AMS websites including some with periodical puzzles. Those problems are a bit more mathematically pure (not all are computational or amenable to solutions by computer); and they're quite a bit more difficult than what I saw on Project Euler. I only recall ever successfully solving one AMS monthly puzzle, and I had a good bit of formal mathematical training. Nimur (talk) 22:06, 2 February 2014 (UTC)

Certainly if you do a web search for something like "interview questions programming algorithms", you'll find a large number of puzzles. I'm not sure there's anything quite like Project Euler out there. You might also check out books; try searching for "algorithm programming puzzles book", which has a number of interesting leads. -- Beland (talk) 18:36, 3 February 2014 (UTC)

Text processing

Hello, I have a few dozen text files, ~700 lines each. They are FORTRAN namelist files, with large blocks of comments, denoted by '!' at the start of each comment line. I'd like to take an input file (or files), and output a "stripped" version, with no comments, and no blank lines. The goal is to reduce the files to ~100 lines each, with (almost) all the same definitions on each line n#, so I can easily do diffs and other processing. I know there are a few ways to do this, but what is the best? I can probably fool around with sed (or awk/grep?), or maybe do it internally in vim_(text_editor). So, my questions:

1. Can anyone help me out with specific sed syntax or vim commands for this task (preferably with a brief explanation of how it works)?

2. Can anyone personally recommend a practical tutorial on regular expressions? I know there are tons out there, and I know the concepts, but I'd like to spend a little time with a good tutorial to (finally) learn it well. I know there are subtle differences in regex syntax and interpretation between systems; I'd prefer the kind that works in vim/sed. Thanks, SemanticMantis (talk) 20:00, 31 January 2014 (UTC)

This website - http://txt2re.com - provides a handy interface to generate regular-expressions by example. You can use it to do the work for you, or you can use it to learn how to construct regular expressions by example. It supports several languages; sed and vim are "perl-like" in their expression syntax (but if you have arbitrarily complex regular expressions, you will undoubtedly find incompatibilities). Nimur (talk) 22:04, 31 January 2014 (UTC)

To strip lines that start with zero or more whitespace characters followed by either a ! or end-of-line, you can pipe the files through perl -ne 'print unless /^\s*(!|$)/'. The regex is delimited by // characters, ^ means the beginning of the line, \s means a single whitespace character, * means zero of more repetitions of the preceding thing, ( | ) means either the thing on the left or the thing on the right, ! is a literal !, and $ means the end of the line. In vim you have to put backslashes in front of the (|) characters, and in POSIX syntax you additionally have to replace the \s with [:space:]. Regular expression#Syntax may be worth reading.

I think there's no reason to learn sed or awk, because Perl is better, but I never learned sed or awk so there may be something I don't know. I no longer use Perl for programs longer than one line since I learned Python, but Perl is still useful for command-line filters like this. -- BenRG (talk) 04:35, 1 February 2014 (UTC)

Thanks Nimur and BenRG! SemanticMantis (talk) 16:22, 3 February 2014 (UTC)

Resolved

Automated download

Here's where I show my total ignorance of Java and related entities. What I would like to do is take the tedium out of downloading a series of files. Let's suppose the main URL is www.[picturesite].com. Say there are 1,000 files there, each with a unique URL, e.g. www.[picturesite].com/picture0001.jpg or some suc. I could individually select and download the files, but that would take a fair chunk of time. So I would like to have some sort of script which would loop from 0001 to 1000, post the resulting URL, and each time do an automated right-click and file-save-as. Is there some way to do this? I'm guessing Java could do this, but I don't know much about Java beyond how to spell it. Thank you for any help you can provide. ←Baseball Bugs ^{What's up, Doc?} carrots→ 20:23, 31 January 2014 (UTC)

If you are already a skilled programmer, you can write new software to perform this task in Java or another language of your choice. There are other ways to accomplish your goal, though:

• You could write a simple script to wrap a standard HTTP transport utility program, like cURL or wget
• You could use a download-management software like DownThemAll for Firefox.

Nimur (talk) 20:56, 31 January 2014 (UTC)

With wget you don't even need a wrapper -- "wget -r" will recursively download the target URL and everything underneath it. Looie496 (talk) 15:27, 1 February 2014 (UTC)

Yes, everything, recursively, and as rapidly as it can. You need to at least use an option to limit the speed, and one of the various options to limit the depth. If you just want to download jpg images linked from a page, you might think that using --accept=jpg would do what you want, but it doesn't because the html index page doesn't have a jpg extension and so never gets downloaded, even though you specified it explicitly on the command line. If you add html to the acceptable extensions, it downloads linked html files as well. If the images are on a different host than the index page, for the same reason there's no way to restrict wget to that host. There's no way to ask wget to parse an already-downloaded html file for links. And there's no clean way to resume the recursive download if it's interrupted for whatever reason. It seems like a theoretically useful tool, but I've never managed to get it to do anything properly except download a single file. I think DownThemAll will probably work better, though I've never used it. -- BenRG (talk) 22:24, 1 February 2014 (UTC)

I agree that 'wget' is the best tool for the job - but be aware that it won't catch everything. What it does is to find all of the links in the HTML, fetch all of those, then look for all of the links in the files that it downloaded - and so forth. However, it's very possible (even quite likely) that a site will create the URL's for the image files on-the-fly. So in that case, wget may be unable to track them all. For example, a page may have HTML code to display a tiny thumbnail of an image - and only fetch the full-resolution version if you click on it. It might well generate the thumbnail and the full-rez image filenames in JavaScript code. I doubt that there is a means to do exactly what the OP requests. However, wget will work in enough cases to be worth trying...especially on simpler websites. SteveBaker (talk) 19:54, 3 February 2014 (UTC)

When I want to download a long list of links that follow a regular pattern, I use spreadsheet software to help me quickly make the list of links, then copy the list and paste it into a text file links.txt. Then I do wget -i links.txt to download the links.

For example, to make 1000 links of the format http://www.[picturesite].com/picture0001.jpg you could do the following:

• Part 1: Make a list of numbers from 1 to 1000

1. In cell A1 enter 1
2. In cell A2 enter =A1+1
3. Click on cell A2 and press Ctrl+C to copy it.
4. In the Name box, enter A3:A1000 to select cells A3 to A1000.
5. Press Ctrl+V to paste.

• Part 2: Embed the numbers in a text list

1. In cell B1 enter ="http://www.[picturesite].com/picture"&TEXT(A1,"0000")&".jpg"
2. Click on cell B1 and press Ctrl+C to copy it.
3. In the Name box, enter B2:B1000 to select cells B2 to B1000.
4. Press Ctrl+V to paste.

• Part 3: Save the list in a text file

1. In the Name box, enter B1:B1000 to select cells B1 to B1000.
2. Press Ctrl+C to copy.
3. Open Notepad and press Ctrl+V to paste. Save the file as links.txt

--Bavi H (talk) 03:22, 4 February 2014 (UTC)

Windows Connect Security

Is it secure to copy a Wi-Fi password to another computer by using the flash drive Windows Connect function? As in, is the information encrypted in any way? I know that on the USB stick, the information seems stored in a regular text file. If it is not, how should I go around connecting the other computers? My network has been compromised often in the past week by people harassing me. I tried searching if any keyloggers were installed, but it doesn't seem so. I have a military grade 63-character WPA2 password, with DD-WRT installed as firmware. — Preceding unsigned comment added by Tretrach (talk • contribs) 22:56, 31 January 2014 (UTC)

I haven't used Windows Connect, but if plugging in the USB stick is enough to transfer the Wi-Fi password—without typing in another (presumably shorter) password that protects it, or otherwise authenticating yourself—then your adversary can do the same thing to get network access if they have the contents of the USB stick. It doesn't matter whether it's encrypted.

What's the evidence that your network was compromised? I think it probably wasn't. If it was, your real concern is whatever malware or other security hole gave the attacker access last time; there's no point changing your password and distributing it by whatever means until you've found and plugged that security hole. I don't think your Wi-Fi password is your problem because people just don't hack into home networks that way. There is malware that gathers credit card numbers and online banking credentials and sends them to the Ukraine, but there's no malware that sends wireless network passwords to someone who lives nearby and can get in range of the network to connect to it. There would be a high risk of getting caught, and access to a wireless network is not much of a prize. Besides, if they have malware on one of your computers, they already have access to your network via that computer.

If I were you I'd pick a shorter password and type it in manually from a piece of paper. 10 random alphanumeric characters is about 60 bits. If your adversary can brute-force 60 bits, your network password is the least of your concerns. The adjective "military-grade" applied to anything cryptographic is usually the sign of a scam (see Warning Sign #7). -- BenRG (talk) 04:13, 1 February 2014 (UTC)