Update Dec 5th 2014: Fixed and updated the code.
Update Mar 9th 2017: Updated the code to handle null values in CSV.
This is a simplified Lua script to read and write CSV files. It assumes the separator character does not exist in one of the values.
I don’t know why I couldn’t find a simple code snippet to do this on the net, but sometimes it’s just faster to write it than to search for it. Hopefully this will save you that time too:
--[[ ------------------------------- Save this file as simplecsv.lua ------------------------------- Example use: Suppose file csv1.txt is: 1.23,70,hello there,9.81,102 x,y,,z 8,1.243,test Then the following ------------------------------------- local csvfile = require "simplecsv" local m = csvfile.read('./csv1.txt') -- read file csv1.txt to matrix m print(m[2][3]) -- display element in row 2 column 3 (102) m[1][3] = 'changed' -- change element in row 1 column 3 m[2][3] = 123.45 -- change element in row 2 column 3 csvfile.write('./csv2.txt', m) -- write matrix to file csv2.txt ------------------------------------- will produce file csv2.txt with the contents: 1.23,70,changed there,9.81,123.45 x,y,,z 8,1.243,test the read method takes 4 parameters: path: the path of the CSV file to read - mandatory sep: the separator character of the fields. Optionsl, defaults to ',' tonum: whether to convert fields to numbers if possible. Optional. Defaults to true null: what value should null fields get. Optional. defaults to '' ]] module(..., package.seeall) --------------------------------------------------------------------- function string:split(sSeparator, nMax, bRegexp) if sSeparator == '' then sSeparator = ',' end if nMax and nMax < 1 then nMax = nil end local aRecord = {} if self:len() > 0 then local bPlain = not bRegexp nMax = nMax or -1 local nField, nStart = 1, 1 local nFirst,nLast = self:find(sSeparator, nStart, bPlain) while nFirst and nMax ~= 0 do aRecord[nField] = self:sub(nStart, nFirst-1) nField = nField+1 nStart = nLast+1 nFirst,nLast = self:find(sSeparator, nStart, bPlain) nMax = nMax-1 end aRecord[nField] = self:sub(nStart) end return aRecord end --------------------------------------------------------------------- function read(path, sep, tonum, null) tonum = tonum or true sep = sep or ',' null = null or '' local csvFile = {} local file = assert(io.open(path, "r")) for line in file:lines() do fields = line:split(sep) if tonum then -- convert numeric fields to numbers for i=1,#fields do local field = fields[i] if field == '' then field = null end fields[i] = tonumber(field) or field end end table.insert(csvFile, fields) end file:close() return csvFile end --------------------------------------------------------------------- function write(path, data, sep) sep = sep or ',' local file = assert(io.open(path, "w")) for i=1,#data do for j=1,#data[i] do if j>1 then file:write(sep) end file:write(data[i][j]) end file:write('\n') end file:close() end ---------------------------------------------------------------------
It runs fine first time, but second time onwards it results in “attempt to call global ‘readCSV’ (a nil value)” error.
What should i do to resolve the problem??
I’m not sure why you encountered the error but there were some errors when transferring the code to the blog post. I’ve updated, cleaned and tested the code so please try it now and let me know if you still have problems with it.
———————-
package.path = “/etc/nginx/?.lua;” .. package.path
require “store_key_script”
ngx.print(package.path)
ngx.print(“::::” .. package.cpath)
local fileData = readCSV(“key_store.txt”)
ngx.print(fileData)
ngx.print(“n————-“);
writeCSV(fileData)
—————————————
local function split(str, sep)
sep = sep or ‘,’
fields={}
local matchfunc = string.gmatch(str, “([^”..sep..”]+)”)
if not matchfunc then return {str} end
for str in matchfunc do
table.insert(fields, str)
end
return fields
end
function readCSV(path, sep)
–tonum = tonum or true
sep = sep or ‘,’
local csvFile = {}
local file = assert(io.open(path, “r”))
for line in file:lines() do
fields = split(line, sep)
— not using tonum now
–if tonum then — convert numeric fields to numbers
— for i=1,#fields do
— fields[i] = tonumber(fields[i]) or fields[i]
— end
–end
table.insert(csvFile, fields)
end
file:close()
return csvFile
end
function writeCSV(data, path, sep)
— just want to print in browser only
sep = sep or ‘,’
–local file = assert(io.open(path, “w”))
for i=1,#data do
for j=1,#data[i] do
if j>1 then ngx.print(sep) end
ngx.print(data[i][j])
end
ngx.print(“n”)
end
–file:close()
return true
end
—————————-
If reload the server it works fine at the first attemp but gives following error in next attempts :
storeKeyScript.lua:6: attempt to call global ‘readCSV’ (a nil value) . Help!!
Forgot to mention this was fixed a few months ago…
Pingback: A simple workflow for deep learning – Mubashir Qasim
It works but I have a problem to read null values in csv file. In fact, I want to applicate mean function for the null data to train them and with this method I can’t do that.
Thanks for pointing this out. The above code now supports null values too.
Hello,
I have a code that prints in Lua console and When I am writing to CSV file its not working
print (“x, y, z, a, b, c, d, e”)
for j=0,i do
print (x[j],”,”,y[j],”,”,z[j],”,”,a[j],”,”,b[j],”,”,c[j],”,”,d[j],”,”,e[j])
end
How I can write those values in CSV … as I copied lua script from Text file and ran it . Its printing the values in console but not in CSV. Can you provide code
It is printing the values to the console because that is what you requesting it to do when use use print.
In order to write to a CSV, you should either use the csvfile.write function as shown in the commented section or use the same method that csvfile.write uses by opening a file for writing, e.g:
local file = assert(io.open(path, “w”))
and then use file.write instead of print
Hello,
under which license can I use this code in my project?
I looked around on your website but didn’t find anything about it.
I hereby place it under a public domain license, and hope it will be used for good…