PY0101EN-4-2-WriteFile.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["<center>\n","    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n","</center>\n","\n","# Write and Save Files in Python\n","\n","Estimated time needed: **25** minutes\n","\n","## Objectives\n","\n","After completing this lab you will be able to:\n","\n","*   Write to files using Python libraries\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2>Table of Contents</h2>\n","<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n","    <ul>\n","        <li><a href=\"https://write/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Writing Files</a></li>\n","        <li><a href=\"https://append/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Appending Files</a></li>\n","        <li><a href=\"https://add/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Additional File modes</a></li>\n","        <li><a href=\"https://copy/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Copy a File</a></li>\n","    </ul>\n","\n","</div>\n","\n","<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"write\">Writing Files</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["We can open a file object using the method <code>write()</code> to save the text file to a list. To write to a file, the mode argument must be set to **w**. Let’s write a file **Example2.txt** with the line: **“This is line A”**\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write line to file\n","exmp2 = '/resources/data/Example2.txt'\n","with open(exmp2, 'w') as writefile:\n","    writefile.write(\"This is line A\")"]},{"cell_type":"markdown","metadata":{},"source":["We can read the file to see if it worked:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Read file\n","\n","with open(exmp2, 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["We can write multiple lines:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write lines to file\n","\n","with open(exmp2, 'w') as writefile:\n","    writefile.write(\"This is line A\\n\")\n","    writefile.write(\"This is line B\\n\")"]},{"cell_type":"markdown","metadata":{},"source":["The method <code>.write()</code> works similar to the method <code>.readline()</code>, except instead of reading a new line it writes a new line. The process is illustrated in the figure. The different colour coding of the grid represents a new line added to the file after each method call.\n"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/WriteLine.png\" width=\"500\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["You can check the file to see if your results are correct\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Check whether write to file\n","\n","with open(exmp2, 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["We write a list to a **.txt** file  as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Sample list of text\n","\n","Lines = [\"This is line A\\n\", \"This is line B\\n\", \"This is line C\\n\"]\n","Lines"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write the strings in the list to text file\n","\n","with open('Example2.txt', 'w') as writefile:\n","    for line in Lines:\n","        print(line)\n","        writefile.write(line)"]},{"cell_type":"markdown","metadata":{},"source":["We can verify the file is written by reading it and printing out the values:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if writing to file is successfully executed\n","\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["However, note that setting the mode to **w** overwrites all the existing data in the file.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'w') as writefile:\n","    writefile.write(\"Overwrite\\n\")\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n","<h2 id=\"Append\">Appending Files</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["We can write to files without losing any of the existing data as follows by setting the mode argument to append: **a**.  you can append a new line as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write a new line to text file\n","\n","with open('Example2.txt', 'a') as testwritefile:\n","    testwritefile.write(\"This is line C\\n\")\n","    testwritefile.write(\"This is line D\\n\")\n","    testwritefile.write(\"This is line E\\n\")"]},{"cell_type":"markdown","metadata":{},"source":["You can verify the file has changed by running the following cell:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if the new line is in the text file\n","\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n","<h2 id=\"add\">Additional modes</h2> \n"]},{"cell_type":"markdown","metadata":{},"source":["It's fairly ineffecient to open the file in **a** or **w** and then reopening it in **r** to read any lines. Luckily we can access the file in the following modes:\n","\n","*   **r+** : Reading and writing. Cannot truncate the file.\n","*   **w+** : Writing and reading. Truncates the file.\n","*   **a+** : Appending and Reading. Creates a new file, if none exists.\n","\n","You dont have to dwell on the specifics of each mode for this lab.\n"]},{"cell_type":"markdown","metadata":{},"source":["Let's try out the **a+** mode:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'a+') as testwritefile:\n","    testwritefile.write(\"This is line E\\n\")\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["There were no errors but <code>read()</code> also did not output anything. This is because of our location in the file.\n"]},{"cell_type":"markdown","metadata":{},"source":["Most of the file methods we've looked at work in a certain location in the file. <code>.write() </code> writes at a certain location in the file. <code>.read()</code> reads at a certain location in the file and so on. You can think of this as moving your pointer around in the notepad to make changes at specific location.\n"]},{"cell_type":"markdown","metadata":{},"source":["Opening the file in **w** is akin to opening the .txt file, moving your cursor to the beginning of the text file, writing new text and deleting everything that follows.\n","Whereas opening the file in **a** is similiar to opening the .txt file, moving your cursor to the very end and then adding the new pieces of text. <br>\n","It is often very useful to know where the 'cursor' is in a file and be able to control it. The following methods allow us to do precisely this -\n","\n","*   <code>.tell()</code> - returns the current position in bytes\n","*   <code>.seek(offset,from)</code> - changes the position by 'offset' bytes with respect to 'from'. From can take the value of 0,1,2 corresponding to beginning, relative to current position and end\n"]},{"cell_type":"markdown","metadata":{},"source":["Now lets revisit **a+**\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'a+') as testwritefile:\n","    print(\"Initial Location: {}\".format(testwritefile.tell()))\n","    \n","    data = testwritefile.read()\n","    if (not data):  #empty strings return false in python\n","            print('Read nothing') \n","    else: \n","            print(testwritefile.read())\n","            \n","    testwritefile.seek(0,0) # move 0 bytes from beginning.\n","    \n","    print(\"\\nNew Location : {}\".format(testwritefile.tell()))\n","    data = testwritefile.read()\n","    if (not data): \n","            print('Read nothing') \n","    else: \n","            print(data)\n","    \n","    print(\"Location after read: {}\".format(testwritefile.tell()) )"]},{"cell_type":"markdown","metadata":{},"source":["Finally, a note on the difference between **w+** and **r+**. Both of these modes allow access to read and write methods, however, opening a file in **w+** overwrites it and deletes all pre-existing data. <br>\n","To work with a file on existing data, use **r+** and **a+**. While using **r+**, it can be useful to add a <code>.truncate()</code> method at the end of your data. This will reduce the file to your data and delete everything that follows. <br>\n","In the following code block, Run the code as it is first and then run it with the <code>.truncate()</code>.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'r+') as testwritefile:\n","    data = testwritefile.readlines()\n","    testwritefile.seek(0,0) #write at beginning of file\n","   \n","    testwritefile.write(\"Line 1\" + \"\\n\")\n","    testwritefile.write(\"Line 2\" + \"\\n\")\n","    testwritefile.write(\"Line 3\" + \"\\n\")\n","    testwritefile.write(\"finished\\n\")\n","    #Uncomment the line below\n","    #testwritefile.truncate()\n","    testwritefile.seek(0,0)\n","    print(testwritefile.read())\n","    "]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"copy\">Copy a File</h2> \n"]},{"cell_type":"markdown","metadata":{},"source":["Let's copy the file **Example2.txt** to the file **Example3.txt**:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Copy file to another\n","\n","with open('Example2.txt','r') as readfile:\n","    with open('Example3.txt','w') as writefile:\n","          for line in readfile:\n","                writefile.write(line)"]},{"cell_type":"markdown","metadata":{},"source":["We can read the file to see if everything works:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if the copy is successfully executed\n","\n","with open('Example3.txt','r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["After reading files, we can also write data into files and save them in different file formats like **.txt, .csv, .xls (for excel files) etc**. You will come across these in further examples\n"]},{"cell_type":"markdown","metadata":{},"source":["Now go to the directory to ensure the **.txt** file exists and contains the summary data that we wrote.\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2> Exercise </h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["Your local university's Raptors fan club maintains a register of its active members on a .txt document. Every month they update the file by removing the members who are not active. You have been tasked with automating this with your Python skills. <br>\n","Given the file currentMem, Remove each member with a 'no' in their Active coloumn. Keep track of each of the removed members and append them to the exMem file. Make sure the format of the original files in preserved.   (*Hint: Do this by reading/writing whole lines and ensuring the header remains* ) <br>\n","Run the code block below prior to starting the exercise. The skeleton code has been provided for you, Edit only the cleanFiles function.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["#Run this prior to starting the exercise\n","from random import randint as rnd\n","\n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","fee =('yes','no')\n","\n","def genFiles(current,old):\n","    with open(current,'w+') as writefile: \n","        writefile.write('Membership No  Date Joined  Active  \\n')\n","        data = \"{:^13}  {:<11}  {:<6}\\n\"\n","\n","        for rowno in range(20):\n","            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n","            writefile.write(data.format(rnd(10000,99999),date,fee[rnd(0,1)]))\n","\n","\n","    with open(old,'w+') as writefile: \n","        writefile.write('Membership No  Date Joined  Active  \\n')\n","        data = \"{:^13}  {:<11}  {:<6}\\n\"\n","        for rowno in range(3):\n","            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n","            writefile.write(data.format(rnd(10000,99999),date,fee[1]))\n","\n","\n","genFiles(memReg,exReg)\n"]},{"cell_type":"markdown","metadata":{},"source":["Start your solution below:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["\n","def cleanFiles(currentMem,exMem):\n","    '''\n","    currentMem: File containing list of current members\n","    exMem: File containing list of old members\n","    \n","    Removes all rows from currentMem containing 'no' and appends them to exMem\n","    '''\n","    \n","    pass \n","\n","\n","# Code to help you see the files\n","# Leave as is\n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","cleanFiles(memReg,exReg)\n","\n","\n","headers = \"Membership No  Date Joined  Active  \\n\"\n","with open(memReg,'r') as readFile:\n","    print(\"Active Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","with open(exReg,'r') as readFile:\n","    print(\"Inactive Members: \\n\\n\")\n","    print(readFile.read())\n","                \n","    "]},{"cell_type":"markdown","metadata":{},"source":["Run the following to verify your code:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["def testMsg(passed):\n","    if passed:\n","       return 'Test Passed'\n","    else :\n","       return 'Test Failed'\n","\n","testWrite = \"testWrite.txt\"\n","testAppend = \"testAppend.txt\" \n","passed = True\n","\n","genFiles(testWrite,testAppend)\n","\n","with open(testWrite,'r') as file:\n","    ogWrite = file.readlines()\n","\n","with open(testAppend,'r') as file:\n","    ogAppend = file.readlines()\n","\n","try:\n","    cleanFiles(testWrite,testAppend)\n","except:\n","    print('Error')\n","\n","with open(testWrite,'r') as file:\n","    clWrite = file.readlines()\n","\n","with open(testAppend,'r') as file:\n","    clAppend = file.readlines()\n","        \n","# checking if total no of rows is same, including headers\n","\n","if (len(ogWrite) + len(ogAppend) != len(clWrite) + len(clAppend)):\n","    print(\"The number of rows do not add up. Make sure your final files have the same header and format.\")\n","    passed = False\n","    \n","for line in clWrite:\n","    if  'no' in line:\n","        passed = False\n","        print(\"Inactive members in file\")\n","        break\n","    else:\n","        if line not in ogWrite:\n","            print(\"Data in file does not match original file\")\n","            passed = False\n","print (\"{}\".format(testMsg(passed)))\n","    \n","\n"]},{"cell_type":"markdown","metadata":{},"source":["<details><summary>Click here for the solution</summary>\n","\n","```python\n","def cleanFiles(currentMem,exMem):\n","    with open(currentMem,'r+') as writeFile: \n","        with open(exMem,'a+') as appendFile:\n","            #get the data\n","            writeFile.seek(0)\n","            members = writeFile.readlines()\n","            #remove header\n","            header = members[0]\n","            members.pop(0)\n","                \n","            inactive = [member for member in members if ('no' in member)]\n","            '''\n","            The above is the same as \n","\n","            for member in members:\n","            if 'no' in member:\n","                inactive.append(member)\n","            '''\n","            #go to the beginning of the write file\n","            writeFile.seek(0) \n","            writeFile.write(header)\n","            for member in members:\n","                if (member in inactive):\n","                    appendFile.write(member)\n","                else:\n","                    writeFile.write(member)      \n","            writeFile.truncate()\n","                \n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","cleanFiles(memReg,exReg)\n","\n","# code to help you see the files\n","\n","headers = \"Membership No  Date Joined  Active  \\n\"\n","\n","with open(memReg,'r') as readFile:\n","    print(\"Active Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","with open(exReg,'r') as readFile:\n","    print(\"Inactive Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","```\n","\n","</details>\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n","<h2>The last exercise!</h2>\n","<p>Congratulations, you have completed your first lesson and hands-on lab in Python. However, there is one more thing you need to do. The Data Science community encourages sharing work. The best way to share and showcase your work is to share it on GitHub. By sharing your notebook on GitHub you are not only building your reputation with fellow data scientists, but you can also show it off when applying for a job. Even though this was your first piece of work, it is never too early to start building good habits. So, please read and follow <a href=\"https://cognitiveclass.ai/blog/data-scientists-stand-out-by-sharing-your-notebooks/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\" target=\"_blank\">this article</a> to learn how to share your work.\n","<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["## Author\n","\n","<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\" target=\"_blank\">Joseph Santarcangelo</a>\n","\n","### Other Contributors\n","\n","<a href=\"https://www.linkedin.com/in/jiahui-mavis-zhou-a4537814a?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2021-01-01\">Mavis Zhou</a>\n","\n","## Change Log\n","\n","| Date (YYYY-MM-DD) | Version | Changed By  | Change Description                  |\n","| ----------------- | ------- | ----------- | ----------------------------------- |\n","| 2020-10-16        | 1.3     | Arjun Swani | Added exercise                      |\n","| 2020-10-16        | 1.2     | Arjun Swani | Added section additional file modes |\n","| 2020-10-16        | 1.1     | Arjun Swani | Made append a different section     |\n","| 2020-08-28        | 0.2     | Lavanya     | Moved lab to course repo in GitLab  |\n","\n","<hr>\n","\n","## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.8.5"}},"nbformat":4,"nbformat_minor":4}