From Newsgroup: comp.lang.forth

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.

TSV solves that problem. Now, what does it require to get supported in 4tH?

https://youtu.be/2lZ72SD-eXg

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

Hans Bezemer <the.beez.speaks@gmail.com> writes:

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.
TSV solves that problem. Now, what does it require to get supported in 4tH?

TSV has the same problem I'd expect. Maybe you want NSV, netstring values.

http://cr.yp.to/proto/netstrings.txt

https://en.wikipedia.org/wiki/Netstring
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

In article <nnd$3db95194$5edbc7b4@7139ac300b44d767>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.

TSV solves that problem. Now, what does it require to get supported in 4tH?

https://youtu.be/2lZ72SD-eXg

If there is a generally accepted standard, one has no choice to abide by
that standard.
I can download my bank data in CSV but TSV is not an option.
So TSV is of limited use.

Problems handling strings is more to be blamed on the lack of
sensible wordsets. With my
$@ $! $+! $C+ $/ $\ $^
wordset, I experience few problems.

E.g. I consume a bank data file and generate a postscript file
with exactly the fields that I want, in the correct order.
(In a fraction of the time struggling with spreadsheets would cost.)

Hans Bezemer

--
The Chinese government is satisfied with its military superiority over USA.
The next 5 year plan has as primary goal to advance life expectancy
over 80 years, like Western Europe.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 16/11/2025 1:12 am, Hans Bezemer wrote:

A well behaved CSV is easy to parse. However, if there are quotes, delimiters or line terminators involved, it gets complicated quickly.
...

Can't say I've had the need - but then I recently found myself having to
parse an assembler include file which I never expected. Never say never? Prompted by your video I've ported your csv parser grateful at not having
to reinvent this particular wheel! Thanks. TSV? That was new to me too.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 16-11-2025 13:04, albert@spenarnc.xs4all.nl wrote:
On 16-11-2025 13:04, albert@spenarnc.xs4all.nl wrote:

If there is a generally accepted standard, one has no choice to abide by that standard.
I can download my bank data in CSV but TSV is not an option.
So TSV is of limited use.

Oh, I agree completely. And as I state in the intro "I don't hate CSV".
It's just I consider it to be a very pragmatic standard - and not an
elegant one. As I try to explain, the "fixes" are so ad hoc, it's just
more of a pain to parse than it should have been.

Problems handling strings is more to be blamed on the lack of
sensible wordsets. With my
$@ $! $+! $C+ $/ $\ $^
wordset, I experience few problems.

As I demonstrated - there are more ways to skin a cat. I did it with a
custom parser. Needless to say, I did not have any problems whatsoever.

E.g. I consume a bank data file and generate a postscript file
with exactly the fields that I want, in the correct order.
(In a fraction of the time struggling with spreadsheets would cost.)

And that's exactly why I made this video. All that extra code takes time
to develop, to debug and to execute. TSV is a simple format that takes
very little code to write, even less code to read (especially when there
the source is vanilla TSV) - and it has been formally codified.

Although I will continue to consider CSV as a standard input format for
my converters, I might consider converting some of it into TSV.
Especially C programs might benefit from this format, since the standard strtok() is a rather dirty function, overwriting the source string and ignoring empty tokens.

That's why I like TSV - IMHO it is much better designed than CSV,

Hans Bezemer

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 17-11-2025 00:48, dxf wrote:

On 16/11/2025 1:12 am, Hans Bezemer wrote:

A well behaved CSV is easy to parse. However, if there are quotes, delimiters or line terminators involved, it gets complicated quickly.
...

Can't say I've had the need - but then I recently found myself having to parse an assembler include file which I never expected. Never say never? Prompted by your video I've ported your csv parser grateful at not having
to reinvent this particular wheel! Thanks. TSV? That was new to me too.

Thanks Ed! It always makes my day when I wrote something useful!

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 17-11-2025 00:48, dxf wrote:

On 16/11/2025 1:12 am, Hans Bezemer wrote:

A well behaved CSV is easy to parse. However, if there are quotes, delimiters or line terminators involved, it gets complicated quickly.
...

Can't say I've had the need - but then I recently found myself having to parse an assembler include file which I never expected. Never say never? Prompted by your video I've ported your csv parser grateful at not having
to reinvent this particular wheel! Thanks. TSV? That was new to me too.

For those interested - this is a non-destructive routine C routine to
parse a TSV record. It resembles WORD a bit. Given you provide the right
size, it's memory safe as well. Empty fields are properly parsed as well.

Expanding escape codes is quite trivial IMHO. By saving "IN" and
comparing it to the updated value of "IN" we can detect a EOL.

Hans Bezemer

#include <stdio.h>

static unsigned In = 0;

char *parse_tsv (char *src, char *dst, size_t n)
{
char *p = dst;

src += In;

while ((*src != '\t') && (*src != '\0') && (n)) {
*dst = *src; In++; dst++; src++; n--;
}

*dst = '\0'; if (*src == '\t') In++;
return (p);
}


int main ()
{
char s[] = "\tThis\t\tis\tthe\tend";
char d[64];
unsigned q;

do {
q = In;
printf ("token=[%s]\n", parse_tsv (s, d, 63));
} while (q != In);
}

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Not really. Watch the video.

Maybe you want NSV, netstring values.

Well, that isn't exactly a data format (like SYLK, DIF, XLS, JSON, XML,
etc.) - rather a string format IMHO.

Hans Bezemer

Hans Bezemer <the.beez.speaks@gmail.com> writes:

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.
TSV solves that problem. Now, what does it require to get supported in 4tH?

http://cr.yp.to/proto/netstrings.txt

https://en.wikipedia.org/wiki/Netstring

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 15-11-2025 23:28, Paul Rubin wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.
TSV solves that problem. Now, what does it require to get supported in 4tH?

TSV has the same problem I'd expect. Maybe you want NSV, netstring values.

http://cr.yp.to/proto/netstrings.txt

https://en.wikipedia.org/wiki/Netstring

I won't say it's my best work, but it'll do. 4tH-isms are provided. This
will deal with netstrings. On error, the NEGATIVE lengths provide
information on the error at hand:

-1 : Missing comma
-2 : Bad number
-3 : Bad length
-4 : Missing colon

---8<---
S" MAX-N" ENVIRONMENT? \ query environment
[IF] \ if successful
NEGATE 1- CONSTANT (ERROR) \ create constant (ERROR)
[ELSE]
.( Warning: ) CHAR ( EMIT .( ERROR) CHAR ) EMIT .( undefined) CR
[THEN]

: STRING CREATE CHARS ALLOT ;
: ;THEN POSTPONE EXIT POSTPONE THEN ; IMMEDIATE
: CHOP 1- SWAP CHAR+ SWAP ; \ 4TH
: ERROR? (ERROR) OVER = ; \ 4TH
: NUMBER ( a n1 -- n2)
0. 2SWAP OVER C@ [CHAR] - = DUP >R
IF CHOP THEN >NUMBER SWAP DROP 0=
IF D>S R> IF NEGATE THEN ELSE 2DROP (ERROR) R> DROP THEN
;

-1 constant N$NOCOMM
-2 constant N$BADNUM
-3 constant N$BADLEN
-4 constant N$NOCOLO

: netstr>
over over 0 swap \ search for colon
0 ?do over i chars + c@ [char] : = if i + leave then loop
tuck 2>r dup 0> if \ colon found
1+ /string 1- \ extract string
2dup chars + c@ [char] , <> if 2rdrop drop N$NOCOMM ;then
2r> number error? if drop drop N$BADNUM ;then
over <> if drop N$BADLEN ;then \ check length
;then 2rdrop drop drop N$NOCOLO \ colon not found
;

: >netstr s" ," 2swap dup s" :" rot s>d <# #s #> ;
: netstr! >r >netstr r@ place r@ +place r@ +place r> +place ;

32 string mynet$
s" Hello world!" mynet$ netstr!
mynet$ count type cr
s" 12:Hello world!," netstr> . . cr
s" 0:," netstr> . . cr depth .

No other words required.
---8<---
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Not really. Watch the video.

Absolutely not! If this was a topic where a video made things really
more obvious than a written explanation (say, the performance of a
gymnastic artist), I would find a pointer to a video acceptable,
although it would still be unlikely that I would watch the video.

But CSV/TSV is not one of these topics. If you want to convince me
that TSVs, whatever they are, have an advantage over CSVs, provide the
argument in writing. I can read. Maybe you can write.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2025 CFP: http://www.euroforth.org/ef25/cfp.html
EuroForth 2025 registration: https://euro.theforth.net/
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Not really. Watch the video.

Absolutely not! If this was a topic where a video made things really
more obvious than a written explanation (say, the performance of a
gymnastic artist), I would find a pointer to a video acceptable,
although it would still be unlikely that I would watch the video.

But CSV/TSV is not one of these topics. If you want to convince me
that TSVs, whatever they are, have an advantage over CSVs, provide the argument in writing. I can read.

- anton

"Maybe you can write." You think? https://thebeezspeaks.blogspot.com/
A 700 page manual. And this:

"Yeah, I thought: let's do something completely different. So here we
are. BTW, if you assumed I hate CSV, you're very wrong. I love CSV. Just
load a line from the file into the Terminal Input Buffer - and let's
start parsing it. If you got a fairly straight forward CSV file, you'll dissect those records with very little effort. Try doing that with JSON
or XML without including a massive library that takes the dirty work out
of your hands.

So no, I'm fine with CSV. It's great. I love it. It's kind to my hands.
It tastes like real coffee. Well, there you have it.

It's also one of the most popular formats for data exchange. Virtually
every single application supports it. Except the example spreadsheet
that came with Borlands Turbo C, version 2. And that's a shame, because
it is much older than you think. The IBM Fortran (level H extended)
compiler under OS/360 already supported CSV. And we're talking 1972
here. And the first version of Turbo C was released in 1987. I'd say
that's plenty of time add some CSV support, don't you think?

It's also referenced in the 1983 SuperCalc manual:, page 7-35: “Use the program Super Data Interchange to convert a SuperCalc data file to a
Comma Separated Values file”. Bummer, but I couldn't get this so-called “SDI” program to produce anything remotely useful. But I digress..

It took a full two decades before it was officially formalized by RFC
4180. In the meanwhile a lot of variations had emerged, especially where
the delimiters were concerned. Comma's were replaced by pipes,
semi-colons and even TABs. According to RFC 4180 the use of these
alternative delimiters is not allowed.

Still, all the European Excels I've encountered use semi-colons rather
than commas, probably because in Europe the comma is used as a decimal separator. Which is annoying if you intend to delimit your file with
commas. Now, fun fact, RFC 4180 offers a solution to that - simply
enclose that data element with double quotes. Which in itself creates
the next problem - what if your data contains double quotes as well?
Next solution - double the sucker up.

The consequence of all this ad hoc patching is, that if you wanna parse
these fields, you can neither parse for commas - nor for quotes. And if
you do, you have to implement all kinds of corrective counter measures
to get it exactly right. But wait - there's more. Another smart move was
to allow raw line terminators in fields. So now, you can't even continue
the parse - no, you gotta save whatever you got, refill the entire
ff-ing buffer again and restart from the beginning. So tell me guys,
when did you design this thing? After working hours? While knocking back
your tenth beer?

Another thing you probably didn't realize, but all CSV files are
actually Windows files. Every line has to be terminated by a “carriage return - line feed” pair.

But enough talk, let's see how it behaves in real life. I've created a spreadsheet with some information about computer screens and exported it
to CSV. Seems harmless enough. Now let's try to read and list it. Note,
we're working with 4-t-H today.

• First we define a word to open the file. The only thing you have to declare is its name and whether the file is read (that is: input) or
written (that is: output);

• What it leaves on the stack is a 4-t-H file handle. If the file
couldn't be opened, it contains an invalid file handle. The word ERROR?
adds a flag to the stack;

• When TRUE, we can call ABORT" to issue an error message and abort;

• If FALSE, ABORT" will be ignored and the file handle will be
duplicated - so it can be USEd. Now all input is sourced from this file.
And yes, it will leave an item on the stack - and later we will see why.
And that's it, we're done;

• Now we got to read this file field by field. So we PARSE the line
using the comma - just like we've seen before. And if it contained data,
we write that string to the console - and we rinse and repeat until
there are no more fields left;

• Now we got to read this file line by line. REFILL reads an entire line
to the input buffer - and returns TRUE when it succeeded. If it
succeeded we ask our previously defined READ-FIELDS to parse that line.
Note we separate each record with an extra carriage-return;

• We're ready to glue all the components together. First we have to
check whether a filename was given on the command line. The program name
is always the first argument - so yes, like in C, you got at least one argument. The second argument would be the data file. Conclusion: we
expect two arguments. The total number of arguments is put on the stack
by the ARGN word;

• So, we expect two arguments. Less is considered an error. In that
case, we call ABORT" to get us out of this mess;

• Our command line arguments are zero-based numbered. So our .CSV file
is at index 1. Now we know for sure there are at least two arguments, we
can retrieve it with the ARGS word. We can feed it to our OPEN-FILE word
and subsequently call READ-RECORDS;

• When we're all done - remember that value we left on the stack? It was
the file handle. So all we got to do is call the CLOSE word - and it
will use that value to close the file. Easy as pie!

And here you got the consequences of .CSV when the going gets tough.
It's butt ugly. Unusable. Now. Can we fix this? Yes, we can. Two
additional lines and one modified one - that's all it takes. Barely an inconvenience.

So - let's address the differences. Yeah, the two additional lines
include the libraries. That's easy. Now, what do they do?

• Well, PARSE-CSV takes quoted fields into consideration. So you get
exactly what's between the quotes;

• But if it also contains double quotes, we're not done yet. That's
where CSV> comes in. That one “undoubles” the quotes in a field. Now
we're done!

• Usually I define a word called FIELD> where I combine these words. And
if needed I can add the filters I deem necessary to perform the task at
hand.

And here you got the result. It's perfectly fine. But of course, the
question rises - could we do better? With less code - and an even easier
to parse format? And the answer is, yes, we can! The IANA registered
such a format in September 2000. Its de facto specification is the text/tab-separated-values media type. But note that fields that contain
tabs or linebreaks are not allowed in this format. That's a bummer,
don't you think?

But the Library of Congress expanded the format, adding a requirement
for escaped tabs, carriage returns, newlines and backslashes. Which made
it much more universal. The now defunct website “dataprotocols.org” also lists a specification, dated May, 2014. According to that same page,
Jason Dusek is the original author. The specification itself is
virtually identical to the one by the Library of Congress. And although
the prominence of these distinguished institutions is not in dispute -
it's not the same as an RFC.

I wanted it. I wanted it badly. Fortunately, I'd already written a CSV
writer for 4-t-H. And since TSV is not that different from CSV, all I
had to do is to write a routine to escape those pesky control characters
- and I was in business. Okay. I escaped a few more. Since unescaping
had to be done by another, already available 4-t-H library that also
unescaped those characters.

And when I was done, what would it take to turn this simple CSV file
reader into a TSV file converter? Just a handful of lines:

• Ok, we got to include the library. That much is clear;

• We take an extra argument - which is the output file;

• Now, here things get a bit different - the TSV writer requires it's
own OPEN-FILE word, called TSVopen. Note - it knows it's an output file
- no need to specify that twice. Yeah - and when we're done, we have to
close it with .. TSVclose! What a surprise! (and no, contrary to OPEN it leaves nothing on the stack);

• Let's fix the parsing routines. This time we won't write anything to
the console, but straight to the TSV file. We write a field using
TSVtype - and we terminate a record using TSVcr. I don't think that one
is rocket science. And we're done!

Now let's try it out and convert our little spreadsheet to TSV. And here
we are! It looks perfectly fine. Let's see if we can read it. Let's take
our original CSV reader and see how much it takes to turn it into a TSV reader. Well, not much. Just change the delimiter, and we're good to go.
Now, since this one does not contain any control characters, we won't
need to expand any escape codes. If we did, we'd need another library.
One we already have. But won't need here.

Anyway, it works just fine. There you go.

BTW, let's assume you need another format. That's not a problem. We got
more libraries. You can convert your file to Wikicode. Or an HTML table.
Or a 4-t-H table. Or JSON. Or LibreOffice. Or MS-Office. Or KSpread. And
yes, even CSV. And they all got roughly the same API - except that
TSVtype is now called XLStype. Yeah, it's all very confusing. I'm so, so sorry.

But anyways - ending on that positive note, I'm Hans Bezemer and this
was “Back and Forth”."

Hans Bezemer

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Maybe you can write.

Gee, I forgot:

- I also wrote one of the most popular tutorials on Forth: https://www.linuxlinks.com/excellent-free-books-learn-forth/
- I was a columnist for fifteen years in three Dutch paper magazines: https://www.agconnect.nl/auteurs/hans-bezemer?page=1
I must have written over 150 columns.

And that's not counting the thousands of pages I've written professionally..

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

Hans Bezemer


--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 11/18/2025 8:22 AM, Hans Bezemer wrote:

- I also wrote one of the most popular tutorials on Forth: https://www.linuxlinks.com/excellent-free-books-learn-forth/

I looked for https://ficl.sourceforge.net/pdf/Forth_Primer.pdf out on archive.org (where I've saved a number of other Forth books under my favorites list) but couldn't find it.

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

Dr. Strangelove I presume.

BTW, who is J. L. Bezmer?

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 18-11-2025 22:24, Buzz McCool wrote:

On 11/18/2025 8:22 AM, Hans Bezemer wrote:

- I also wrote one of the most popular tutorials on Forth:
https://www.linuxlinks.com/excellent-free-books-learn-forth/

I looked for https://ficl.sourceforge.net/pdf/Forth_Primer.pdf out on archive.org (where I've saved a number of other Forth books under my favorites list) but couldn't find it.

Try: https://thebeez.home.xs4all.nl/ForthPrimer/2FP.html

It was intended to be a document people could freely include with their compiler - and several did. However, if they cease development
altogether and the site disappears ..

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

Dr. Strangelove I presume.

Correct!

BTW, who is J. L. Bezmer?

That's me if you add an "e" at the right place. ;-)
Dutchie thing. Our formal first names are usually Latin!

E.g. my brother is Rudolphus Theodorus. Believe it or not..

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

In article <nnd$03c928fa$6ca4ab00@183c17b0b12e73e2>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Maybe you can write.

Gee, I forgot:

- I also wrote one of the most popular tutorials on Forth: >https://www.linuxlinks.com/excellent-free-books-learn-forth/
- I was a columnist for fifteen years in three Dutch paper magazines: >https://www.agconnect.nl/auteurs/hans-bezemer?page=1
I must have written over 150 columns.

And that's not counting the thousands of pages I've written professionally..

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

So you can write. Do it. Write me a nice 10 line summary why TSC is
better than CSV. Like Anton I can't be bothered to view a video that
promise to take 10 minutes of my remaining life.

Hans Bezemer

Groetjes Albert

--
The Chinese government is satisfied with its military superiority over USA.
The next 5 year plan has as primary goal to advance life expectancy
over 80 years, like Western Europe.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

In article <nnd$3db95194$5edbc7b4@7139ac300b44d767>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

A well behaved CSV is easy to parse. However, if there are quotes,
delimiters or line terminators involved, it gets complicated quickly.

You have cited the final solution in the RFC.

- Separate records with &,.
- If the string contains &, quote it by double quotes &".
_ Inside a string double the &" . 1)

1) This is the system I use for quotes strings in ciforth, it goes
back Algol68 and it is genius.

This is a final good solution. Only two characters to take care off.

"He said: ""You are not smart.""" TYPE
is valid, but gives an syntax error on a system that is not double quoting, luckily.

TSV solves that problem. Now, what does it require to get supported in 4tH?

https://youtu.be/2lZ72SD-eXg

Hans Bezemer

Groetjes Albert
--
The Chinese government is satisfied with its military superiority over USA.
The next 5 year plan has as primary goal to advance life expectancy
over 80 years, like Western Europe.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 19-11-2025 11:29, albert@spenarnc.xs4all.nl wrote:

In article <nnd$03c928fa$6ca4ab00@183c17b0b12e73e2>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Maybe you can write.

Gee, I forgot:

- I also wrote one of the most popular tutorials on Forth:
https://www.linuxlinks.com/excellent-free-books-learn-forth/
- I was a columnist for fifteen years in three Dutch paper magazines:
https://www.agconnect.nl/auteurs/hans-bezemer?page=1
I must have written over 150 columns.

And that's not counting the thousands of pages I've written professionally.. >>
"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

So you can write. Do it. Write me a nice 10 line summary why TSC is
better than CSV. Like Anton I can't be bothered to view a video that
promise to take 10 minutes of my remaining life.

Last thing. I'm not "gekke Henkie":

CSV is simple but its rules for commas, quotes, and newlines create
heavy ambiguity.

Variations in delimiters across regions make CSV parsing inconsistent
and brittle.

Embedded quotes require doubling, adding more edge cases for every
parser to handle.

CSV fields may contain raw line breaks, forcing complex multi-buffer
parsing logic.

Because of these quirks, CSV readers must implement layers of corrective patches.

TSV avoids most issues because tabs rarely appear naturally in data fields.

With Library of Congress extensions, TSV cleanly escapes tabs and newlines.

TSV files are predictable: fields split cleanly and escaping rules are unambiguous.

TSV parsers need far less code, making them easier to build, test, and maintain.

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV.

Hans Bezemer

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

In article <nnd$44f7c84d$7e431930@4e0d5b1886fec755>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 19-11-2025 11:29, albert@spenarnc.xs4all.nl wrote:

In article <nnd$03c928fa$6ca4ab00@183c17b0b12e73e2>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Maybe you can write.

Gee, I forgot:

- I also wrote one of the most popular tutorials on Forth:
https://www.linuxlinks.com/excellent-free-books-learn-forth/
- I was a columnist for fifteen years in three Dutch paper magazines:
https://www.agconnect.nl/auteurs/hans-bezemer?page=1
I must have written over 150 columns.

And that's not counting the thousands of pages I've written professionally..

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

So you can write. Do it. Write me a nice 10 line summary why TSC is
better than CSV. Like Anton I can't be bothered to view a video that
promise to take 10 minutes of my remaining life.

Last thing. I'm not "gekke Henkie":

I admit that my comment is crude. Your efforts to publish video's
is commendable and its content are worthwhile. However there is
a communication overload going on, and most people appreciate
an overview before engaging to view it.

I try to give a good example.

The first page of my pdf doc is a section called overview. It fits on
one page.
You can download the archive, open the pdf, and you can decide
whether it is for you, and discard it when it is still in the download directory.
(If you print the postscript, the features are on the cover page.)

----------------------------------------------

Forth is an interactive programming system. ciforth is a family of
Forth's that can be generated in many different version for many
different operation systems. It is sufficiently close to the ISO
standard to run most programs intended to be portable. It deviates
where less used features where objectionable to implement. *Note
Manual::, Section Portability.
......
......
......
These are the features:

All ciforth's are “case sensitive” . This is version 6.0.0. It is running in protected mode. It is running under Linux Blocks are
allocated in files. A number has a precision of 64 bits. It calls
linux system from Forth directly. It has compiler security, sacrificing
some ISO compatibility. It use ‘PP’ instead of the ISO ‘>IN’ . It contains the full ISO CORE in the kernel, more than is needed to make it
self contained. It contains a field in the header to point to source.
It is indirect threaded.

----------------------------------------------------------------

I recently looked into revaforth. It took a terrible amount of time
to arrive at information that it was a kind of Forth. That much I
guessed from the name.

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data fields. Shit, it is based on tabs? That damns it in my view. Wish I knew that

earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV. Agree to disagree. Newlines are invisible, but at least you know they are there. tabs are the silent killers.

Hans Bezemer

Groetjes Albert
--
The Chinese government is satisfied with its military superiority over USA.
The next 5 year plan has as primary goal to advance life expectancy
over 80 years, like Western Europe.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

Am 20.11.2025 um 11:48 schrieb albert@spenarnc.xs4all.nl:

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data fields.

Shit, it is based on tabs? That damns it in my view. Wish I knew that earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV.

Agree to disagree. Newlines are invisible, but at least you know they are there. tabs are the silent killers.

FWIW Excel can export tables directly in TSV format. This can sometimes
be very useful, especially when you get number series with commas as
decimal points.
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 20-11-2025 11:48, albert@spenarnc.xs4all.nl wrote:

In article <nnd$44f7c84d$7e431930@4e0d5b1886fec755>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 19-11-2025 11:29, albert@spenarnc.xs4all.nl wrote:

In article <nnd$03c928fa$6ca4ab00@183c17b0b12e73e2>,
Hans Bezemer <the.beez.speaks@gmail.com> wrote:

On 18-11-2025 16:03, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

On 15-11-2025 23:28, Paul Rubin wrote:

TSV has the same problem I'd expect.

Maybe you can write.

Gee, I forgot:

- I also wrote one of the most popular tutorials on Forth:
https://www.linuxlinks.com/excellent-free-books-learn-forth/
- I was a columnist for fifteen years in three Dutch paper magazines:
https://www.agconnect.nl/auteurs/hans-bezemer?page=1
I must have written over 150 columns.

And that's not counting the thousands of pages I've written professionally..

"MEIN FÜHRER, I CAN WRITE!" (bonus question: which movie is this?)

So you can write. Do it. Write me a nice 10 line summary why TSC is
better than CSV. Like Anton I can't be bothered to view a video that
promise to take 10 minutes of my remaining life.

Last thing. I'm not "gekke Henkie":

I admit that my comment is crude. Your efforts to publish video's
is commendable and its content are worthwhile. However there is
a communication overload going on, and most people appreciate
an overview before engaging to view it.

First, thanks for the compliment. It is appreciated. However, I think
you're oversimplifying some things.

Like so many things I've created, I make them because I like viewing,
reading or using them myself. If others like 'em as well, first I thank
you very much for appreciating my work - and second, I congratulate you
for having good taste. ;-)

But frankly, my intended audience was primarily myself. I don't know if
you remember when I presented 4tH (some 30 years ago), but the reception
was not too welcoming. But what you guys never got is that I made that
thing for myself.

And don't understand me wrong, I cherish every help I've gotten through
the years on this adventure. But it would have happened anyway - even if
I'd gotten 0 downloads up till this day.

Second, and most important - not every document is created equal. A
casual history on Forth is not the same as a glossary or a tutorial. And
it's structure needs to reflect that purpose. E.g. some docs - even non-fiction - are just there for entertainment. They don't need to give
an overview, or be searchable.

Like I said, I did columns for quite a while. A good column builds up to
a conclusion like a fire cracker builds up to an explosion. Usually,
every statement carries with it a burden of proof - not a column. You
don't have space for that - and it doesn't contribute to its primary
purpose. Worse, it doesn't even have to be true. As long as you are
surprised, get this "Aha-Erlebnis", makes you change your mind or get
you to say "I never thought of that" it served its purpose.

So - in short, your statements are only true for a limited class of
documents. Likewise, my vids are not primarily reference works. They're primarily made to entertain, with a hint of mild education. And that as
an answer to all those horrible "men behind the terminal" YT's.

If those are for you, fine. They're not for me. Hence, I make my own.
And if you've not understood the message so far - I'll continue to make
them until I lose interest.

I recently looked into revaforth. It took a terrible amount of time
to arrive at information that it was a kind of Forth. That much I
guessed from the name.

Not all programmers are able to write accessible information. It's a
gift. :-)

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data fields.

Shit, it is based on tabs? That damns it in my view. Wish I knew that earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV.

Agree to disagree. Newlines are invisible, but at least you know they are there. tabs are the silent killers.

You should have watched the video. Then you'll find your criticism is completely unfounded. Which is completely understandable. If summaries
were really sufficient, why would we need the complete thing?

Hans Bezemer

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 20-11-2025 12:18, minforth wrote:

Am 20.11.2025 um 11:48 schrieb albert@spenarnc.xs4all.nl:

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data
fields.

Shit, it is based on tabs? That damns it in my view. Wish I knew that
earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative
to CSV.

Agree to disagree. Newlines are invisible, but at least you know they are
there. tabs are the silent killers.

FWIW Excel can export tables directly in TSV format. This can sometimes
be very useful, especially when you get number series with commas as
decimal points.

You're completely right. IIRC, they call it "*.txt" files. Libreoffice
only offers what are technically "CSV with tabs".

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

Hans Bezemer <the.beez.speaks@gmail.com> writes:

Libreoffice
only offers what are technically "CSV with tabs".

I just opened libreoffice calc, typed in a few cells, then told it to
"save as", then selected "Text CSV (.csv)", and by default it gave me
',' as field separator and '"' as quote. The output looks as follows:

123,abc

So Libreoffice also offers CSVs in their original meaning, and offers
to use other separators and quotes. The character encoding can
apparently be chosen, too.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2025 CFP: http://www.euroforth.org/ef25/cfp.html
EuroForth 2025 registration: https://euro.theforth.net/
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 20-11-2025 19:44, Anton Ertl wrote:

Hans Bezemer <the.beez.speaks@gmail.com> writes:

Libreoffice
only offers what are technically "CSV with tabs".

I just opened libreoffice calc, typed in a few cells, then told it to
"save as", then selected "Text CSV (.csv)", and by default it gave me
',' as field separator and '"' as quote. The output looks as follows:

123,abc

So Libreoffice also offers CSVs in their original meaning, and offers
to use other separators and quotes. The character encoding can
apparently be chosen, too.

Again, if you had watched the video (or read the script you requested)
you would have known that [a] a LOC TSV is *not* what Libreoffice
provides and [b] every CSV *without* "," as delimiter is *technically*
not a (RFC 4180) CSV - since it doesn't comply to that standard.

But true, it can be considered a IANA TSV - provided it doesn't double
quotes. It's late, I haven't tested this yet. But I'll report back in
the morning.

Hans Bezemer

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 21/11/2025 1:20 am, Hans Bezemer wrote:

On 20-11-2025 12:18, minforth wrote:

Am 20.11.2025 um 11:48 schrieb albert@spenarnc.xs4all.nl:

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data fields.

Shit, it is based on tabs? That damns it in my view. Wish I knew that
earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV.

Agree to disagree. Newlines are invisible, but at least you know they are >>> there. tabs are the silent killers.

FWIW Excel can export tables directly in TSV format. This can sometimes
be very useful, especially when you get number series with commas as
decimal points.

You're completely right. IIRC, they call it "*.txt" files. Libreoffice only offers what are technically "CSV with tabs".

PlanMaker (SoftMaker Office) offers a 'txt' format in which field and text separators default to tab and none respectively. Whether it precisely duplicates Excel's 'txt', I don't know.

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 20-11-2025 23:26, Hans Bezemer wrote:

But true, it can be considered a IANA TSV - provided it doesn't double quotes. It's late, I haven't tested this yet. But I'll report back in
the morning.

I tested it. {Tab} denotes a [CHAR] 9:

Brand{Tab}Model{Tab}Size
HP{Tab}Series 3 Pro{Tab}"23,8"""
Dell{Tab}P2425H{Tab}"23,8"""
Lenovo{Tab}L15{Tab}"15,6"""
Verbatim{Tab}Portable Screentouch{Tab}"15,6"""
Verbatim{Tab}Portable Ultra{Tab}"17,3"""
Acer{Tab}EK2510Gbi{Tab}"24,5"""
Iiyama{Tab}XUB2493HS-B6{Tab}"23,8"""
ASUS{Tab}ZenScreen MB166C{Tab}"15,6"""
Dell{Tab}P2425{Tab}"24,1"""

Since it uses not only double quotes, but also doubles the native
quotes, it does *not* comply to either IANA mime type, nor RFC 4180 (LOC
is a superset of IANA):

Now, let's assume we use "comma" as a delimiter and save the *entire raw records* from above as single fields (AKA this thing has just ONE field
- and forget about the quotes), this would have been written in TSV LOC format:

Brand\tModel\tSize
HP\tSeries 3 Pro\t"23,8"""
Dell\tP2425H\t"23,8"""
Lenovo\tL15\t"15,6"""
Verbatim\tPortable Screentouch\t"15,6"""
Verbatim\tPortable Ultra\t"17,3"""
Acer\tEK2510Gbi\t"24,5"""
Iiyama\tXUB2493HS-B6\t"23,8"""
ASUS\tZenScreen MB166C\t"15,6"""
Dell\tP2425\t"24,1"""

Where "\t" denotes [CHAR] \ + [CHAR] t. Hence, since LOC requires *all*
fields to have certain (control) characters escaped (like [CHAR] 9)
there can *never* be a conflict between a delimiter and an embedded TAB
in a field.

If one would have done their research instead of just assuming things,
they would have saved me to explain their misgivings. Or one could just
have watched the video where I summarize all this, having done the
research for you. ;-)

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 21-11-2025 02:59, dxf wrote:

On 21/11/2025 1:20 am, Hans Bezemer wrote:

On 20-11-2025 12:18, minforth wrote:

Am 20.11.2025 um 11:48 schrieb albert@spenarnc.xs4all.nl:

<SNIP>

TSV avoids most issues because tabs rarely appear naturally in data fields.

Shit, it is based on tabs? That damns it in my view. Wish I knew that
earlier. And you commented that newlines are present in CSV fields?

<SNIP>

Overall, TSV offers a simpler, safer, and more reliable alternative to CSV.

Agree to disagree. Newlines are invisible, but at least you know they are >>>> there. tabs are the silent killers.

FWIW Excel can export tables directly in TSV format. This can sometimes
be very useful, especially when you get number series with commas as
decimal points.

You're completely right. IIRC, they call it "*.txt" files. Libreoffice only offers what are technically "CSV with tabs".

PlanMaker (SoftMaker Office) offers a 'txt' format in which field and text separators default to tab and none respectively. Whether it precisely duplicates Excel's 'txt', I don't know.

Most text exports actually save a CSV variant - unless there are no
quotes - or embedded carriage returns, line feeds or tabs in any of the fields, effectively resulting in a IANA TSV. BTW, aforementioned
characters plus backslashes need to be escaped for LOC TSV compliance.

I think most spreadsheets effectively save a CSV variant - although
PlanMaker seems to get pretty close.

LOC writes about the issue of CSV variants:

"Several relatively common variations from the strict form specified by
RFC 4180 are found and may be supported by software tools such as those
listed below as Useful References:

1. In locales where the comma character is used in place of a decimal
point in numbers, the separator between fields/columns is often a semicolon.
2. The line break character may be CR or LF, not necessarily CRLF.
3. Some Unix-based applications may use a different escape mechanism for indicating that one of the separator characters occurs within a text
value. The individual character is preceded by a backslash character
rather than enclosing the entire string in double quotes.
4. Single quotes may be treated as equivalent to double-quotes for
escaping (also known as "text-qualification")."

In short, in spite of RFC 4180 there is still a wide variety of CSV
variants, which may overlap with either IANA or LOC TSVs.

And yeah, I'm guilty as well - technically writing a variant rather than
a full RFC 4180 compliant format ;-)

Now I think of, I may make that configurable.

Hans Bezemer



--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.lang.forth

On 21-11-2025 14:13, Hans Bezemer wrote:

On 20-11-2025 23:26, Hans Bezemer wrote:

But true, it can be considered a IANA TSV - provided it doesn't double
quotes. It's late, I haven't tested this yet. But I'll report back in
the morning.

I tested it. {Tab} denotes a [CHAR] 9:

Brand{Tab}Model{Tab}Size
HP{Tab}Series 3 Pro{Tab}"23,8"""
Dell{Tab}P2425H{Tab}"23,8"""
Lenovo{Tab}L15{Tab}"15,6"""
Verbatim{Tab}Portable Screentouch{Tab}"15,6"""
Verbatim{Tab}Portable Ultra{Tab}"17,3"""
Acer{Tab}EK2510Gbi{Tab}"24,5"""
Iiyama{Tab}XUB2493HS-B6{Tab}"23,8"""
ASUS{Tab}ZenScreen MB166C{Tab}"15,6"""
Dell{Tab}P2425{Tab}"24,1"""

Since it uses not only double quotes, but also doubles the native
quotes, it does *not* comply to either IANA mime type, nor RFC 4180 (LOC
is a superset of IANA):

Now, let's assume we use "comma" as a delimiter and save the *entire raw records* from above as single fields (AKA this thing has just ONE field
- and forget about the quotes), this would have been written in TSV LOC format:

Brand\tModel\tSize
HP\tSeries 3 Pro\t"23,8"""
Dell\tP2425H\t"23,8"""
Lenovo\tL15\t"15,6"""
Verbatim\tPortable Screentouch\t"15,6"""
Verbatim\tPortable Ultra\t"17,3"""
Acer\tEK2510Gbi\t"24,5"""
Iiyama\tXUB2493HS-B6\t"23,8"""
ASUS\tZenScreen MB166C\t"15,6"""
Dell\tP2425\t"24,1"""

Where "\t" denotes [CHAR] \ + [CHAR] t. Hence, since LOC requires *all* fields to have certain (control) characters escaped (like [CHAR] 9)
there can *never* be a conflict between a delimiter and an embedded TAB
in a field.

If one would have done their research instead of just assuming things,
they would have saved me to explain their misgivings. Or one could just
have watched the video where I summarize all this, having done the
research for you. ;-)

Hans Bezemer

BTW, I just made my CSV writer configurable upto RFC 4180, by making the delimiter configurable - and consistently writing CR+LF as EOL.

.. and of course [CHAR] 9 should be just "9". That's its ASCII code.
[CHAR] 9 results in ASCII code 57 - which is not a TAB. That will learn
me to dabble in BASIC and then go Forth. ;-)

Hans Bezemer
--- Synchronet 3.21a-Linux NewsLink 1.2