Best way to get the last nth lines from a large text file

[AUGE_OHR]

Question: is there no hb_FBof() and then hb_FSkip() -1 ?

this is from Xbase++ Help File

Code: Select all

FUNCTION F_Use( cFileName )
FUNCTION F_GoTop( aFile )
FUNCTION F_GoBottom( aFile )
FUNCTION F_Skip( aFile, nBytes )
FUNCTION F_Read( aFile, nBytes )
FUNCTION F_Bof( aFile )
FUNCTION F_Eof( aFile )

FSEEK.ZIP

(856 Bytes) Downloaded 52 times

[edk]

Edward,

Question: is there no hb_FBof() and then hb_FSkip() -1 ?

Serge

Serge,
I don't see the hb_FBof () function, it's hb_FEof () but it returns status if end of file has been reached, so I used hb_FGoBottom () -> Go to the last record in a text file.
https://vivaclipper.wordpress.com/2014/ ... functions/ (thanks to my dearly departed friend Esgici)

Edward.

[serge_girard]

Edward,

I meant that a hb_FBof() is missing so we cannot read text file from bottom to top with skip-1, just like dbf file...

Code: Select all

use somefile
go bottom
do while .not. bof()
   //some commands
   skip -1
enddo   

Your solution works perfectly!

Serge

[edk]

Edward,

I meant that a hb_FBof() is missing so we cannot read text file from bottom to top with skip-1, just like dbf file...

Code: Select all

use somefile
go bottom
do while .not. bof()
   //some commands
   skip -1
enddo   

Your solution works perfectly!

Serge

Oh, of course - you're right Serge.
But we can work around it another way:

Code: Select all

hb_FGoBottom ()
Do While .T.
	/* line processing */
	If hb_FRecno () = 1
		Exit
	EndIf
	hb_FSkip ( -1 )
EndDo

[HGAutomator]

martingz,

Thanks, but no. I don't want to load the entire file into memory as a string. As edk demonstrated, the preference is to open the file, then jump to the last line:

hb_FGoBottom ()

, then skip upwards, without reading anything yet:

hb_FSkip ( nLinesToRead * (-1))



Regards,

[franco]

This seems to work, Thanks to all above.
There could be a better way to copy to new file, but I could not figure out. So I used the old way.
Give a try with your big file. You mentioned sometimes you may want to a different number of records
Here you can enter the number of records to skip.
You could also enter a spot to name the file for new text.

Code: Select all

#include "hmg.ch"
#include "FILEIO.CH"

Function Main()
	CRTEMP()
Return

Function CrTemp
Local  YOURFILE := 'YOURFILENAME.TXT'          //PUT YOUR FILE NAME HERE
Local yourrec := 0 , yourrec1 := 0, c1Line := "", prow := 0
yourrec := val(alltrim(inputbox('Enter Number Of Records To Skip Up From Bottom')))
HB_Fuse(YOURFILE)
c1Line:= HB_FLastRec()-(yourrec-1)       
HB_FGoBottom()                           
if HB_FLastRec()  > yourrec
 	yourrec1:= HB_FlastRec()-yourrec+1
	HB_FGoto(yourrec1) 
else
	HB_FGoTop()
endif
c1Line := HB_FReadLN() 
SET PRINTER TO 'NEWFILE.TXT'
SET DEVICE TO PRINT
SET PRINT ON

DO WHILE ! HB_FEof()
	@ Prow+1,1 SAY c1Line
	HB_FSkip()
	c1Line := HB_FReadLN()
	Prow:=Prow+1
	loop
ENDDO
SET DEVICE TO SCREEN
SET PRINTER TO
HB_FGoBottom()
HB_FUse()
RETURN

Franco

[franco]

Revised, added filename and check for output file name.

Code: Select all

#include "hmg.ch"
#include "FILEIO.CH"

Function Main()
	CRTEMP()
Return

Function CrTemp
Local  newfilename := 'NewFile.txt', YOURFILE := '6305.txt'          //PUT YOUR FILE NAME HERE
Local yourrec := 0 , yourrec1 := 0, c1Line := "", prow := 0
yourrec := val(alltrim(inputbox('Enter Number Of Records To Skip From Bottom')))
if yourrec < 1
	yourrec := 0
endif
newfilename := alltrim(inputbox('Enter New file Name','EXAMPLE:NewFile.txt'))
if len(alltrim(newfilename)) < 1
	newfilename:= 'NewFile.txt'
endif
HB_Fuse(YOURFILE)      
HB_FGoBottom()    
if HB_FLastRec()  > yourrec
 	yourrec1:= (HB_FlastRec()+1)-(yourrec)
         HB_FGoto(yourrec1) 
else
	HB_FGoTop()
endif
c1Line := HB_FReadLN() 
SET PRINTER TO &NewFileName
SET DEVICE TO PRINT
SET PRINT ON

DO WHILE ! HB_FEof()
	@ Prow,1 SAY c1Line
	HB_FSkip()
	c1Line := HB_FReadLN()
	Prow:=Prow+1
	loop
ENDDO
SET DEVICE TO SCREEN
SET PRINTER TO
HB_FGoBottom()	
HB_FUse()
RETURN

[HGAutomator]

Hi,

Is there a more reliable way to determine eof() for text files, other than HB_FEof()? It's returning .T., even though I can confirm that we're not at the last record.

Btw, I deleted the post that complained about a subtraction. I misread the result. Everything is fine, as far as the calcluation goes.

In the case of HB_FEof(), the file is open, and we're able to read the line, and we're able to determine the hb_FRecNo(), which is 10000 less than the last record number/total number of lines.

HB_FEof() returns .T., as soon as we HB_FUse() the file. It remains .T., when we skip a record, and of course when we're at the end of file.

Is there another alternative function? Or should I just check the current record number, and compare it to HB_FlastRec()?

[HGAutomator]

Got it. Per the suggestions at

https://groups.google.com/g/harbour-use ... 2NTQERAgAJ

, I replaced HB_FEof() with hb_FAtEof(), and everything works as expected. In fact, the file is generated MUCH faster than I thought it would.

[franco]

I had to modify again. Look at last post I am going to modify. We need a skip after finding your rec.
I added a msgbox to check.