C# - File Handling
by Sam AllenProblem
Handle file IO effectively in your application. Utilize the .NET framework's powerful C# methods on the File static class, and reduce the maintenance burden by simplifying and shortening code. Maintain or establish good performance.
Solution: C#
My experience is that C# provides excellent file handling/IO methods. They are optimized in the framework so that you don't need to hand-optimize buffer sizes or other mechanics. Make sure to include the IO namespace, like this:
// // Include this namespace for all the examples. // using System.IO;
File static class. The table that's next shows a bunch of the most useful File methods we can use. Many C# programmers use these methods quite extensively, particularly the ones dealing with lines and text.
| Method name | Use |
| File.ReadAllBytes |
Useful for files not stored as plain text. See example near the bottom. |
| File.ReadAllLines |
Microsoft: "Opens a file, reads all lines of the file with the specified encoding,
and closes the file." See the benchmark below. |
| File.ReadAllText |
Returns the contents of the text file at the specified path as a string. See the benchmark below. |
| File.WriteAllBytes |
Not covered here. It should be used in conjunction with File.ReadAllBytes. |
| File.WriteAllLines | Stores a string array in the specified file, overwriting the contents. |
| File.WriteAllText | Writes the contents string to a text file. |
| File.AppendAllText |
Use to append the contents string to the file at path. Microsoft: "Appends the specified string to the file, creating the file if it doesn't already exist." |
| File.AppendText |
Not covered here. You must use standard StreamWriter code. |
Read in lines with File.ReadAllLines
Here you want to read all the lines in from a file and place them in an array or List. (For List, see further down in the document.) The following code reads in each line in the file "file.txt" into an array. This is efficient code, but I have performance metrics later on.
//
// Read in every line in specified file.
// This will store all lines in an array in memory,
// which you may not want or need.
//
string[] lines = File.ReadAllLines("file.txt");
foreach (string line in lines)
{
//
// Do something with line
//
if (line.Length > 80)
{
// Example code (disregard).
}
}
Read lines with StreamReader ReadLine
This method is not part of the File static class, but it is in the System.IO namespace. I provide it here as a comparison to the File.ReadAllLines method. Next, I provide some data about how the two methods to read lines perform on different sized files.
//
// Read in every line in the file.
//
// Note how |using| wraps the StreamReader.
// The condition in |while| is also very important to get right.
//
using (StreamReader reader = new StreamReader("file.txt"))
{
string line;
while ((line = reader.ReadLine()) != null)
{
//
// Do something with line just for example.
//
string[] parts = line.Split(',');
}
}
Benchmark results
I hope never to lead my readers to make a reverse optimization in their code. I want people to write code that is easy-to-understand and also very fast and resource-friendly. The following two graphs show how the above two code blocks perform.
| Method and version | Number of lines | Time per iteration (ms) |
| File.ReadAllLines | 52,930 | 28.266 |
| StreamReader ReadLine | 52,930 | 17.534 |
| File.ReadAllLines | 20 | 0.487 |
| StreamReader ReadLine | 20 | 0.480 |
Summary of results. StreamReader is much faster for large files with 10,000+ lines, but the difference for smaller files is negligable. As always, plan for varying sizes of files, and use File.ReadAllLines only when performance isn't critical.
Should I use File.ReadAllText?
Usually, yes. However, I wanted to resolve whether File.ReadAllText was performing well. To answer this, I benchmarked it against StreamReader and found that on a 4 KB file it was almost 40% slower.
void Example()
{
//
// A.
// Read in file with File class.
//
string text1 = File.ReadAllText("file.txt");
//
// B.
// Read in file text with helper method and StreamReader.
// 40% faster!
//
string text2 = ReadFileString("file.txt");
}
static string ReadFileString(string path)
{
//
// Use StreamReader to consume the entire text file.
//
using (StreamReader reader = new StreamReader(path))
{
return reader.ReadToEnd();
}
}
StreamReader helper. In some projects, it would be worthwhile to use the above ReadFileString custom static method. In my project that opens hundreds of small files, it saves 0.1 milliseconds per file. (The numbers were 155 ms and 109 ms for 1000 files.)
What comes next? Keep reading, because I have a bunch of useful methods to work with the File static class. The previous charts should diffuse any worries about performance, at least for smaller files. (If you have huge files, performance is much more important.)
Use List with File.ReadAllLines
List and ArrayList are extremely useful data structures for us programmers, as they allow us to rapidly expand (or shrink) large (or small) 'collections' of objects. Here I show how you can use LINQ to get a List of lines from a file in one line.
//
// Read in all lines in the file,
// and then convert to List with LINQ.
//
List<string> fileLines = File.ReadAllLines("file.txt").ToList();
Count lines with File.ReadAllLines
Do you need to count the lines in a file? Every developer has had this requirement at some time. Are there 10 or 10,000 lines in the web server log file? We don't need to write ten lines of code to do this. Simply reference the Length property.
//
// Another method of counting lines in a file.
// This is NOT the most efficient way, and it
// counts empty lines.
//
int lineCount = File.ReadAllLines("file.txt").Length;
See if a line exists in a file
Does a line containing a specific string exist in the file? Maybe you want to see if a name or location exists in a line in the file. Here again we can harness the power of LINQ to find any matching line.
// // One way to see if a certain string is a line // in the specified file. Uses LINQ to count elements // (matching lines), and then sets |exists| to true // if more than 0 matches were found. // bool exists = (from line in File.ReadAllLines("file.txt") where line == "Some line match" select line).Count() > 0;
Persist data with File.WriteAllLines
When you are done with your in-memory processing, you often need to write the data to disk (persist it). Fortunately, the File class offers an excellent WriteAllLines method. It receives the file name (path) and then the array to write. (This will replace all the file contents!)
//
// Write a string array to a file.
//
string[] stringArray = new string[]
{
"cat",
"dog",
"arrow"
};
File.WriteAllLines("file.txt", stringArray);
// File contains:
// cat
// dog
// arrow
Append text to files
The previous example will replace the file's contents, but for a log file or error listing, we must append to the file. (Sure, we could read in the file, append to that in memory, and then write it out completely again. But that's slow.)
//
// Append text to a file. This method will create a new file if one isn't already there.
// No newlines will be automatically added (add those yourself if wanted).
//
File.AppendAllText("file.txt", "test");
// File contains:
// test
File.AppendAllText("file.txt", "more");
// File contains:
// testmore
Read in all bytes
Finally here, I want to show how you can use File.ReadAllBytes. In my project I use this to cache an image in memory for performance. This works very well and greatly outperforms reading in the image each time.
static class ImageCache
{
static byte[] _logoBytes;
public static byte[] Logo
{
get
{
//
// Returns logo image bytes.
//
if (_logoBytes == null)
{
_logoBytes = File.ReadAllBytes("Logo.png");
}
return _logoBytes;
}
}
}
Conclusion
When using C#, always strive to use the framework as much as possible, and in the most effective way possible. This is more important than many other changes we could make. My experience is that C# and .NET is excellent with file handling, and one benchmark I have seen measures it as even faster than C++ on Windows.