之前其实碰到过很多次了,也大概知道一些,但一直都没有进行总结,所以这次在搜索了很多方法之后进行个大体的总结,方便以后查阅:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import time, os, sys
fname="C:/Python27/NEWS.txt"
#fname=sys.argv[1].strip()
def timeo(fun, n=10):
start = time.clock()
for i in xrange(n): fun()
end = time.clock()
thetime = end-start
return fun.__name__, thetime #返回函数名称,函数运行10次总时间的 元组
def linecount_wc():
return int(os.popen('wc -l nuc').read().split()[0]) #使用外部系统命令 wc -l
def linecount_1( ): #使用方法1
return len(open(fname).readlines())
def linecount_2(): #使用方法2
count = 0
for line in open(fname).xreadlines(): count += 1
return count
def linecount_3(): #使用方法3
count = 0
thefile = open(fname)
while 1:
buffer = thefile.read(65536)
if not buffer: break
count += buffer.count('n')
return count
def linecount_4(): #添加的方法4
count=0
for line in open(fname):
count+=1
return count
for f in linecount_1, linecount_2, linecount_3, linecount_4:
print f.__name__, f()
for f in linecount_1, linecount_2, linecount_3, linecount_4:
print "%s: %.2f" % timeo(f)
'''
linecount_1 4929090
linecount_2 4929090
linecount_3 4929090
linecount_4 4929090
linecount_1: 17.92
linecount_2: 18.14
linecount_3: 9.84
linecount_4: 18.42
但我在EditPlus和Notepad++中打开时,显示的行数分别为:4929060 / 4929090 为什么会有点差距……
'''
起源于啄木鸟社区的一个讨论:http://wiki.woodpecker.org.cn/moin/PyCkBk-4-7,然后我是在“Python:统计文件行数”中找到的计时代码,不过现在Python有timeit模块了,所以这个就自己看着办了,怎么方便/顺手怎么来就好了。
总结:在内存不是太多的情况下,一行一行的读取然后计数其实是一个最快的方法。笨方法不意味着低效!只要能达到目的就行。
#!/usr/bin/env python
# coding=utf-8
# method.1
count = len(open(filepath,'rU').readlines())
count2 = len(linecache.getlines(filepath))
# method.2
count = -1
for count, line in enumerate(open(filepath, 'rU')):
pass
count += 1
#method.3
count = 0
thefile = open(filepath, 'rb')
while True:
buffer = thefile.read(8192*1024)
if not buffer:
break
count += buffer.count('n')
thefile.close()
#method.4
def linecount_w( ):
return int(os.popen('wc -l "C:Python27NEWS.txt"').read().split()[0])
《“用Python统计文件行数的几个方法总结”》 有 1 条评论
C语言编程实现——获取文件行数大小(!feof + fgetc)
https://stackoverflow.com/questions/5139213/count-number-of-line-using-c
https://stackoverflow.com/questions/12733105/c-function-that-counts-lines-in-file
`
FILE *fp = fopen(“myfile.txt”);
int ch;
int count = 0;
do
{
ch = fgetc(fp);
if(ch == ‘\n’) count++;
} while( ch != EOF );
printf(“Total number of lines %d\n”,count);
`