Rathinagiri wrote:So the problem is the append from command.
huiyi_ch wrote:Thank your solved method,But I try use "Harbour MiniGUI Extended Edition build 16.10" the append command ,it can be execute correctly.
Yes, APPEND FROM DELIMITED does not work in HMG as expected with Unicode data files, with or without a UTF-8 BOM. And it is also true that APPEND FROM DELIMITED in MiniGui Extended works as expected. But MiniGui Extended cannot display Unicode, while HMG does. You need Unicode to display Chinese characters.
The HB_ATOKENS approach works well, as long as you do not use quotes around the field values (which APPEND FROM DELMITED allows), and you have a file that MEMOREAD can handle properly. I've tested MEMOREAD for files up to 4G and it seemed to work OK. I don't know about file sizes larger than this.
The HB_ATOKENS call that Rathi gave splits a file up into lines. Since there is only one field per line, the whole line can be read into the field. But if you are reading more than one field per record, you will have to split the line up with another HB_ATOKENS call. The FOR loop would then have to be something like this:
Code: Select all
for i := 1 to hmg_len( aData )
aFields := HB_ATOKENS( aData[ i ], "," )
append blank
replace field1 with aFields[ 1 ]
replace field2 with aFields[ 2 ]
...
next
The data file would look something like this:
Code: Select all
ABC,ABCDEFGHI
DEFG,KLMNOPQ
XY,THIRDREC
You can also use APPEND FROM SDF. I tried this, and it does work as expected with Unicode data files. But if you have multiple fields per line, the lines must be set up differently. You do not use commas, but instead use spaces to pad out the fields to the field length. So if you had two fields, FIELD1 with length 5 and FIELD2 with length 10, an SDF file would have to look like this:
Code: Select all
ABC ABCDEFGHI
DEFG KLMNOPQ
XY THIRDREC
Field lengths in DBF files are number of bytes, not number of characters. Each Chinese character in the PEOPLES1.txt file that you posted takes up 3 bytes. I believe that most Chinese characters in common use take up 3 bytes. Other Chinese characters may take 4 bytes.
The PEOPLES1.txt file that you posted contains a BOM (byte order mark). This does not work with APPEND FROM SDF. It worked only when I removed the BOM. The BOM is usually not necessary in Harbour, including PRG and other source files.