Wikipedia:Reference desk/Archives/Computing/2014 May 5

Computing desk
< May 4	<< Apr | May | Jun >>	May 6 >

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

May 5

Can files disappear when moving them onto a flash drive?

Can files being moved off a pc and onto a flash drive disappear from the pc but not reappear on the flash drive without you having received error messages? — Preceding unsigned comment added by 216.207.8.170 (talk) 06:30, 5 May 2014 (UTC)

They've never done that for me, but, in theory, it is possible if the control software in the flash drive is malfunctioning, and reports a successful write to the computer's operating system when in fact the write was faulty. Another possibility is that the write was successful but a section of the flash drive has become unstable. In this case, file recovery software might (or might not) recover lost files. Dbfirs 08:53, 5 May 2014 (UTC)

This is also a symptom of some cheap generic or counterfeit flash drives produced in China that have a fake capacity. These are low capacity drives (typically 4GB or 8GB) that have been hacked to report a greatly inflated capacity (16GB up to 512GB, sometimes more) to the operating system. Once the true capacity is exceeded, data is quietly lost or corrupted. 24.254.222.118 (talk) 22:53, 5 May 2014 (UTC)

It can happen if you remove the flash drive from the computer before the files are completely written. Flash drives can be slow, so to speed things up, the operating system will cache the data, basically making a note of "when I've got time, finish writing this to the drive". Selecting "eject", "safely remove", or whatever your operating system calls it tells the OS to do the writing as quickly as possible, and let you know when it's done so you can remove the drive without losing data. --Carnildo (talk) 02:33, 8 May 2014 (UTC)

You can lose data this way, but you should get a warning when you unplug the drive ("Delayed Write Failed" on Windows), and there should be truncated version of the file on the flash drive (after repairing the file system). I don't see how a file could disappear completely and silently. -- BenRG (talk) 08:03, 8 May 2014 (UTC)

Yes, the operating system shouldn't lose track of a moving file because it checks the copy before deleting the original. If the write was incomplete, then the original should still be on the PC. Dbfirs 08:16, 8 May 2014 (UTC)

For valuable files, I usually copy to flash drive, then check that the files are readable there, or copy to backup elsewhere, before deleting the originals. I've never lost files in moving them to a flash drive, but, for various reasons mentioned above, sometimes it pays to be over-cautious. The problem with just moving is that the files cannot be restored from the recycle bin if something goes wrong (though in theory they still exist on the PC hard drive until the space is overwritten). "Move" is really a "copy" followed by a "delete". My operating system (Windoze) does not perform the "delete" until it thinks it has checked the existence of the moved files, but when it does delete as part of a "move", it simply removes the file table links without recording the data in the recycle bin and without protecting the space on the hard drive. (Other operating systems might behave differently.) Dbfirs 08:09, 8 May 2014 (UTC)

Hexapawn

I tried to build a "sweet computer" to play the game of Hexapawn as per these instructions (pdf), but I'm having trouble understanding the instructions. Page 7 of the pdf shows that there are two possibilities for the computer's first move. Now, having played several games, I've worked out which is the right first move for the computer. But this means that the game will never get to the stage where I have to work out which is the right second move. What am I doing wrong? Many thanks, --Viennese Waltz 15:43, 5 May 2014 (UTC)

It looks like an interesting game. If you read the first paragraph of the pdf "One of the most exciting branches of computer science is called machine learning" so I would expect machine learning to play a role. The instructions do say on page 3 that a possible move should be picked at random. The way the learning for the computer seems to happen is in the eating sweets corresponding to the last moving move. As you repeat the game it will not be able to repeat that move so bad moves are avoided. To appreciate how the machine learns you should not really try to second guess the computers moves, while there are still un-eaten sweets for each state always pick one at random. --Salix alba (talk): 16:28, 5 May 2014 (UTC)

Are you doing the 3x3 version ? If so, it looks like you can do a brute force approach and just create a tree of every possible game. From there, each player would choose the move on the tree which offers the greatest chance of winning. This method looks like it might work for the 4x4 version too, but 5x5 may well be too much for it. StuRat (talk) 23:16, 5 May 2014 (UTC)

You might be interested in finding a copy of "We Built our own Computers" by A.B.Bolt (1966). I think there is a 2010 re-issue and there are pdfs of it on the web. I'm lucky enough to own an original. There is a section about a matchbox computer that plays Hexapawn and it might help you. When I was a spotty faced schoolboy in 1974, a couple of us made a machine inspired by the JACKIE computer featured. I hope one day to recreate it. --TrogWoolley (talk) 03:14, 6 May 2014 (UTC)

What is Half Compiled Code in .Net?

In .NET programming language like C#, VB.NET, etc;get compiled differently but does not compile into executable code instead it compiles into an intermediate code called Microsoft Intermediate Code Language (MSICL)which is a common language for both the languages. It is also called Half-Compiled Code since it cannot be fully decoded. It is then passed to an Interpreter called Common Language Runtime (CLR). Merlin Thomas (talk) 17:15, 5 May 2014 (UTC)Merlin Thomas-

You appear to be answering your own question. Why? AndyTheGrump (talk) 18:06, 5 May 2014 (UTC)

What is Compiler and Interpreter in .Net Framework?

1.JIT is the internal compiler of .NET which takes MicroSoft Intermediate Code Language (MSICL) code from CLR and executes it to machine specific instructions whereas CLR works as an engine its main task is to provide MSICL code to JIT to ensure that code is fully compiled as per machine specification.
2.Common Language Runtime(CLR) is interpreter while Just In Time(JIT) is compiler in .Net Framework.

Merlin Thomas (talk) 17:15, 5 May 2014 (UTC)Merlin Thomas-

You appear to be answering your own question. Why? AndyTheGrump (talk) 18:06, 5 May 2014 (UTC)

Recovering data when a hard drive crashes

I have a question about recovering data when a hard drive crashes. I don't know much about computers and all the technical jargon, so please keep answers at an elementary level. Thanks. So, here is my question. My hard drive crashed. I sent it to a data recovery service center. They told me that they were able to recover the data. Now, my data contains many hundreds (if not, thousands) of files and folders. Obviously, I cannot recall the name of every single file and folder that I have had (over the years). When I receive my recovered data, how will I know if it is all there? Or if even some of it is missing? In other words, how do I know if the service center recovered 100% of my data ... or perhaps only 50% or 10% or whatever? The tech rep on the phone said – essentially – that I am the only person who would know whether all my files are there or not. He said that the recovery team just recovers what they can recover ... and they don't know if there is any data that they "missed" or were not able to recover. I think he said something like: if files are corrupted, they themselves don't even know that the files exist; they don't "show up". So, is there any way to know the percentage of how many files have been recovered and how many became lost (other than my memory or recollection of the names of my thousands of files)? I kept pressing the repair tech for a "better" answer; he simply kept saying that they have no way of knowing how much (and what) data I originally had on the hard drive before it crashed; I am the only person who will know that. He did offer this: he said, "your hard drive holds 64 gigs; we recovered 50 gigs". Does that mean that my original hard drive had only 50 gigs of data and 14 gigs of free space? Or does that mean that my hard drive originally had 64 gigs of data; they recovered 50 gigs; and the remaining 14 gigs are lost? I am totally confused. Can anyone offer any insight on this? Many thanks. Joseph A. Spadaro (talk) 19:12, 5 May 2014 (UTC)

In general there is no way to know. Say you have a 100 page notebook and your dog chews it up. Careful reassembly and photography lets you reconstruct 86 pages with stuff written on them. The other 14 pages are chewed up beyond hope of recovery. So did they have stuff written on them or were they blank? You can't tell. 70.36.142.114 (talk) 19:59, 5 May 2014 (UTC)

Thanks. Not the answer I wanted to hear, of course. But, that leads me to two follow up questions. (Question 1) In your example, at least you know that there are 14 missing pages. Whether these pages were blank or had writing, you don't know. What is the analogy to the hard drive? Shouldn't they be able to say how many "pages" are missing ... without specifying if the "missing pages" had data or were empty/blank? And (Question 2) so, does the customer just blindly pay the service? I am paying them $ XXXXX, and I have no idea how much of my data they are recovering? It could be 100% or it could be 5% ... and I am still paying them the same flat fee of $ XXXX? Is that how this works? It seems like you are paying a lot of money, and you don't know exactly what you are getting in return ... no? Thanks. Joseph A. Spadaro (talk) 20:06, 5 May 2014 (UTC)

Was there super valuable stuff on the drive? Do you know what happened with the drive, i.e. did it physically crash? Raw data blocks (equivalent to pages in a notebook) aren't intrinsically associated with files. Instead, a few of the blocks (to first approximation, call this the "directory") says which blocks belong to which files. Blocks also get reallocated when you delete files and create new ones. So your partial recovery can result from some blocks having been physically unreadable, but it's also possible that some blocks were readable and yet not associated with files because the file metadata in the directory was itself clobbered somehow (this happens with some drives due to power failures and the like). It's also possible that the blocks were in a file and then you deleted the file, which typically erases the directory entry but leaves the raw blocks alone (they get rewritten when they're reallocated to another file). You can sometimes recover content from the raw blocks if you have an image of the physically recoverable blocks, a reasonably good idea of what you're looking for, some scripting ability, and a lot of time on your hands. It was more reasonable to do manually in the era of 1 megabyte floppy discs than it is with 50 gigabyte hard drives.

Data recovery is expensive, partly because of special equipment and parts that the recovery places have, but (it seems to me) partly because there is obscure knowledge involved, particularly on the hardware side.

No they can't guarantee results. If you are hit by a bus and you go in for surgery, your doctors can't guarantee (cue various jokes) that you'll play the violin normally afterwards, but they do a lot of work on you and you'll get billed for it either way. 70.36.142.114 (talk) 20:16, 5 May 2014 (UTC)

Added: if 64gb was the physical capacity of the hard drive and they recovered 50gb, you did pretty good. If it was something like a 500gb drive and they said you had 64gb of data and they recovered 50gb, they probably figured out the 64gb from the metadata and it's remotely possible that you could piece together some of the rest by content-specific means (digital forensics might have some useful references). Usually recovery goes by first imaging the physically readable blocks from the crashed drive to another drive, then trying to reconstruct files and directories from the raw image using various kinds of software tools. The imaging phase involves special hardware and knowledge that I mentioned: there are some scary youtube videos with conflicting advice, showing people taking drives apart and unsticking heads, and that type of thing. That's where those guys make their money. The software phase, it seems to me, takes some patience and experience, but the tools they use aren't terribly magical and equivalent tools can often be downloaded from the internet.

If you want to keep working on it you could ask about getting the raw sector image that the recovery guys were able to make, if they made one and kept it. You could hire consultants to work on it with you, who would explain what they were doing and use your knowledge of the drive contents to possibly get a little bit more data than the generic recovery did, but if you don't even know what files you had, I don't see how it's worth the expenditure. I lost several boxes of "treasured" personal possessions in a move some years ago, and was upset til I realized that since I couldn't really remember what was in the boxes, they must not have been that important after all, if that makes any sense to you. 70.36.142.114 (talk) 20:43, 5 May 2014 (UTC)

Thanks. In all honestly, I hardly understood a word that you said. Too technical and complicated for my basic understanding. Nearly all of that went over my head, unfortunately. But, to answer your question. Yes, it was very valuable. It was all of my files and folders and data from the last 20+ years. I could hire a second consultant, but I don't even know if there are missing files to be found (in the case that the first service recovered 100% of the data). So, is the bottom line that I can never know and will never know whether all or only part of my data has been recovered? When they do their physical work with the hard drive, they can't even "know" (determine) that there is data that they were unable to access? In other words, can't they determine something like: "sector 12 of the hard drive was corrupted, but all the other sectors are fine" ... ? Which will give me some indication of the extent of lost data. Thanks. Joseph A. Spadaro (talk) 21:09, 5 May 2014 (UTC)

I'd be more optimistic. As they have recovered 50GB they have had a pretty high success ratio. If it was mechanical failure which did not actually damage the actual disk platters then they may have been able to get everything. If some block got corrupted then that can be enough to stop things running but not enough to do significant damage to all your personal files. Its relatively rare to damage a large number of blocks. --Salix alba (talk): 21:22, 5 May 2014 (UTC)

Yes, they should be able to give you a complete list of the sectors they couldn't read, as well as a raw disk image containing all of the sectors that they could read. And they should certainly be able to answer your totally reasonable question about the meaning of "your hard drive holds 64 gigs; we recovered 50 gigs". Call them back and demand an answer. Eventually they will transfer you to somebody who knows something. -- BenRG (talk) 22:11, 5 May 2014 (UTC)

They should be able to tell you "sector 12 was unreadable". But if sector 12 was part of the "directory" (file allocation table etc) then they will no longer know that it described a file that lived at sectors 108241, 697, 30824, 797979, and 50038 in that order. That means there is the possibility that 108241, 697, 30824, 797979, and 50038 are all recoverable but they have no way to know whether they belonged to actual files. If those sectors contain ordinary text you could maybe say "oh that's my wonderful letter from Mom!" and figure out what order they were supposed to be in. But if they were chunks of compressed video, they look basically random and it's near hopeless.

If you had 20 years worth of data on a 64gb disk, that suggests you migrated the data from older and smaller discs and other places, since discs of the mid-1990's were much smaller than 64gb. Is there some chance you can put your hands on the original places where the data came from?

If you've got a consultant working with you, it might be easier for him/her to ask the right questions of the recovery place about getting the images, than for you to figure out exactly what to ask.

How old was this hard drive and what OS was it? There's an issue with the FAT filesystem (basically Windows 98 and earlier, I think) that the blocks in a file are chained by block numbers directly in the blocks, so if the middle block of a 100 block file is corrupted, it can be difficult for recovery to figure out where the rest of the blocks are. But if that happens they should be able to detect the dangling blocks with software and say "here's this 50 block fragment we picked up, it's likely to be the tail end of some file, but we don't know for sure and we don't know which file". That might or might not do you any good. 70.36.142.114 (talk) 22:39, 5 May 2014 (UTC)

Added: I've actually never heard of a 64gb hard drive. 40gb, 60gb, 80gb, 120gb were standard sizes. Do you know the make and model of the drive you had recovered? 70.36.142.114 (talk) 22:40, 5 May 2014 (UTC)

Also, no it's not always possible to tell corruption from good data. They try to read a block and one of three things happens: 1) they read it successfully and it has good data; 2) it is unreadable because the disk surface has been physically damaged; 3) they read it successfully but the data is no good. They can tell you if #2 happens but it's harder for them to distinguish between #1 and #3. Like if they recover a page from your written notebook with stuff written on it. If it is readable English it's probably good. If it's random-looking gibberish, maybe it's because the page was corrupted, but maybe it's because random-looking gibberish is actually what you wrote on the page and it's a successful recovery of that page. Big hard drives are often full of compressed data such as video, and compressed data looks quite random if you haven't got the whole file in one piece. What were the most important types of files on the disk if you don't mind my asking? (I don't mean about the content, I just mean at the level of "photos", "text files", "spreadsheets", etc. since that will affect the ease of identifying files with software.) 70.36.142.114 (talk) 22:47, 5 May 2014 (UTC)

Thanks. I have no idea of the name/model of the hard drive. I guess that it is whatever came standard with my Dell computer. The OS is Windows 8.1. These are files that I have had for 20+ years. I have always backed up my files on, say, a weekly basis. Then, I got lazy. The last time that I did a back up was approximately December 2013. Meaning that four months of new data and new files had never been backed up on my external hard drive. These files, 99% of them, are Word files, Excel files, PDF files, maybe a few photographs. Mainly Office documents (Word, Excel). Thanks. Joseph A. Spadaro (talk) 23:05, 5 May 2014 (UTC)

I just looked around in my Dell computer, under the "PC Information" and such. They actually have four or five hard drives listed! So, I don't know which is the relevant one? Joseph A. Spadaro (talk) 23:08, 5 May 2014 (UTC)

I looked some more. For my "C" Drive, it says "OS (C:)", then it says 2 TB. The only place where I see the value 64 mentioned is where it says "64-bit Operating System". This is all Greek to me!?!?!?! Joseph A. Spadaro (talk) 23:21, 5 May 2014 (UTC)

1) If there are several drives listed I'd say look in all of them for useful info. 2) Wait, if you have a 4 month old backup then this is about 4 months worth of data rather than 20 years, right? If the recovery operation got most of the 4 months back, you're doing pretty good and can look into ways of recreating whatever is missing if it's important. 3) A Dell computer with Windows 8.1 sounds pretty recent, so the data must have gotten onto it from someplace else, like an older computer. So if you still have the old computer with its disk intact, your data from before the migration should still be there.

External hard disks aren't that great a backup strategy by the way: they tend to fail by sitting on the shelf for too long. If the amount of data you're accumulating is growing relatively slowly then you could back up to an external drive daily (a USB flash stick might be big eonugh, and much more convenient) and burn a CD or DVD once a week or so. There are probably also some fully automatic Windows backup tools like AirPort Time Capsule that Apple sells. You just install the backup application on your computer and put the backup machine in your closet, and it takes care of everything for you, backing up all your files over wifi while you sleep at night. Or if you have fast internet and don't mind cloud backup (privacy concerns etc.) you could look at Backblaze. Cheap monthly fee, completely automatic backups over the network, and if your drive crashes and it's too much data to download, they'll put your data on a physical hard drive and ship it to you for a reasonable charge. 70.36.142.114 (talk) 00:58, 6 May 2014 (UTC)

Thanks. From what I can gather, my C Drive is 2 TB. I am not sure why the guy said to me: "out of 64 gigs, we recovered 50 gigs". The computer is brand new, less than a year old. I have data from 20+ years ago. Up until December 2013, all of that data was backed up (external hard drives). The last time I did a back up to my external hard drive was December 2013. So, there is only 4 months of data that is potentially missing (December 2013 through April 2014). Everything, pre-December 2013 is safe and sound on the external hard drive from my most recent back up. After this crisis happened, I went out and subscribed to not one, but two, cloud back up services. I had always been meaning to, and never got around to it. Until this happened. Joseph A. Spadaro (talk) 04:52, 6 May 2014 (UTC)

Also, the data from my last back up (December 2013) was about 8 gigs. I am referring only to files and folders. I don't back up the OS or the program files. I only back up the Word documents, Excel spreadsheets, PDF files, photos, etc. On my December 2013 back up, all of those data files amounted to just about 8 gigs. In the last 4 months, I doubt that I accumulated much more than that. Joseph A. Spadaro (talk) 04:56, 6 May 2014 (UTC)

Thanks, all. Joseph A. Spadaro (talk) 16:56, 7 May 2014 (UTC)

It might be a good idea to do a list dump of all the files on your hard drive to a text file (Command Prompt, "dir /S" in your main folder) to a USB stick every so often; that way you have a list of everything on your drive and can refer back to it if need be. -- 140.202.10.134 (talk) 20:19, 7 May 2014 (UTC)

Ahhh, great idea. I did not know about that. Thanks. Joseph A. Spadaro (talk) 23:53, 7 May 2014 (UTC)