Discussion:
Which newsgroup for json parsing?
Add Reply
Josef Möllers
2024-05-08 13:04:56 UTC
Reply
Permalink
Hi all,

I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1,
"tF":106.0}}

I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.

Thanks in advance,

Josef
Anton Shepelev
2024-05-08 13:21:46 UTC
Reply
Permalink
Post by Josef Möllers
I am trying to parse a json string received through MQTT
from a "Shelly Plug S", e.g.
{"id":0,
"source" : "button",
"output" : true,
"apower" : 0.0,
"voltage": 237.9,
"current": 0.000,
"aenergy":{
"total" : 0.000,
"by_minute": [0.000,0.000,0.000],
"minute_ts": 1715172119},
"temperature":{
"tC": 41.1,
"tF": 106.0}}
I have reformatted your example, because it was a mess.
JSON is so easy to parse, that writing your own parser may
prove an interesting exercise for a couple of days. I used
the following site for reference:

<https://www.json.org/json-en.html>
Post by Josef Möllers
I am trying to use libjson-glib but I can't figure out
what to use as the first argument to
json_gobject_from_data()!
Have you checked the documentation?--

<https://gnome.pages.gitlab.gnome.org/json-glib/func.gobject_from_data.html>

Additionally, the WIKI page has some examples:

<https://wiki.gnome.org/Projects/JsonGlib>
Post by Josef Möllers
I am also looking at libjson-c but cannot find any
examples that could guide me.
Have you checked the documentation?--

<http://json-c.github.io/json-c/json-c-current-release/doc/html/index.html#using>

Observe that examples are optional in that they illustrate
documentation rather then provide it. It is better to learn
from first principles than from examples. See also:

<http://json-c.github.io/json-c/json-c-current-release/doc/html/index.html#gettinghelp>
--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
Lawrence D'Oliveiro
2024-05-23 03:03:07 UTC
Reply
Permalink
Observe that examples are optional in that they illustrate documentation
rather then provide it.
“You can never have too many examples.”
-- Jerry Pournelle
Malcolm McLean
2024-05-08 14:33:48 UTC
Reply
Permalink
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
JSON parsing is the sort of thing you can easily do in C, but you really
don't want to do yourself, becuase it is a lot of fiddly code to write,
and everyone wants to solve exactly the same problem, which is to query
text formatted as JSON, usually for fields whose names and types are
known, but with facilities to list fields onthe fly.

But of course I wrote my own JSON parser in C++, As programmers often
do. But it was at work, so I can't post a copy here, since it was work
for hire and not mine to give away.

Now libjson-glib strikes me as rather a poorly designed library with too
many dependencies, and over complicated.

But here's the prototype for the function

GObject*
json_gobject_from_data (
GType gtype,
const gchar* data,
gssize length,
GError** error
)

Obviously, this creates a Gobject* that you query, from text that you
pass in (he data argument). For some reason the string is not
nul-terminated and so you pass in the length, and you also get a parse
error object, and to be fair this can be extremely important.

But what is the gtype parameter?

WEll a bit of digging reveals this palaver

GType
my_struct_get_type (void)
{
static GType boxed_type = 0;

if (boxed_type == 0)
{
boxed_type =
g_boxed_type_register_static (g_intern_static_string ("MyStruct"),
(GBoxedCopyFunc) my_struct_copy,
(GBoxedFreeFunc) my_struct_free);

json_boxed_register_serialize_func (boxed_type, JSON_NODE_OBJECT,
my_struct_serialize);
json_boxed_register_deserialize_func (boxed_type, JSON_NODE_OBJECT,
my_struct_deserialize);
}

return boxed_type;
}


So you set it up. Ir's a highly flexible way of registering a data
structure at run time. But not what you want to parse a bit of JSON. And
if there's a simple default type to pass in, they are not documenting
this clearly. So if I had a choice, I would simply avoid the library and
choose another.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
Anton Shepelev
2024-05-08 16:53:07 UTC
Reply
Permalink
Post by Malcolm McLean
But here's the prototype for the function
GObject*
json_gobject_from_data (
GType gtype,
const gchar* data,
gssize length,
GError** error
)
Obviously, this creates a Gobject* that you query, from
text that you pass in (he data argument). For some reason
the string is not nul-terminated and so you pass in the
length
I think it is because the function is meant to be universal,
and usable not only with strings but also with buffers.
--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
jak
2024-05-09 13:18:11 UTC
Reply
Permalink
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1,
"tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Josef
Hi,
from your post I can't understand if you are writing
production code or just for yourself, nor if you intend to
deepen your knowledge of json or just acquire the data of
interest from the json. If you are interested in the data and
the code is for you, I can suggest an inelegant but effective
way unless you want to monitor the data in a fast loop. I
would suggest you let the jq command line do the work for
you and grab its output via popen. jq is a command for *nix
but you can also find it for windows. here is an example:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <locale.h>

int main()
{
FILE *fp;
char from_plug[] = "{\"id\":0, \"source\" : \"button\", \"output\"
: true, \"apower\" : 0.0, "
"\"voltage\": 237.9, \"current\": 0.000,
\"aenergy\":{ \"total\" : 0.000, "
"\"by_minute\": [0.000,0.000,0.000],
\"minute_ts\": 1715172119}, "
"\"temperature\":{ \"tC\": 41.1, \"tF\": 106.0}}";
char echo_fmt[] = "cmd /c echo %s | jq
.temperature.tC,.voltage,.source";
size_t cmd_len = snprintf(NULL, 0, echo_fmt, from_plug) + 1;
char cmd[cmd_len];
double temp, volt;
char source[20];

snprintf(cmd, cmd_len, echo_fmt, from_plug);

if((fp = popen(cmd, "r")) != NULL)
{
fscanf(fp, " %lf %lf %*[\"]%[^\"] ", &temp, &volt, source);
setlocale(LC_ALL, "");
wprintf(L"\n"
L"temperature: %6.2lf\u00B0\n"
L"voltage: %6.2lf v\n"
L"source: %hs\n", temp, volt, source);
pclose(fp);
}
else
perror("popen");

return 0;
}


output:

temperature: 41.10°
voltage: 237.90 v
source: button
Josef Möllers
2024-05-13 09:07:25 UTC
Reply
Permalink
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
F'up to self, also to answer a few questions.

First of all, I have switched to libjson-c5 The code I'm using looks
like this:
/* Requires: libjson-c-dev */
# include <json.h>
# include <json_tokener.h>
:
static void
xlate(unsigned int plug, unsigned int sw, const char *payload, char *item)
{
struct json_tokener *tok;
struct json_object *obj1, *obj2;

tok = json_tokener_new();
obj1 = json_tokener_parse_ex(tok, payload, strlen(payload));
if (json_object_object_get_ex(obj1, item, &obj2) && obj2 != NULL)
{
json_bool b;
enum json_type t;
switch (t = json_object_get_type(obj2))
{
case json_type_boolean:
b = json_object_get_boolean(obj2);
:

Anton referred to the many sites that have documentation.
Yes, you're absolutely right. There are many sites that have
documentation, but I found that they were not easy to spot. It usually
depends upon my state of mind AND the correct queries to send to Aunt
Google. I am usually glad for some code examples that help me up the
first step and where I can build upon. I have myself written a few
magazine articles and a lot of internal documentation for my employers
on various programming and architectural topics that aim at people who
start working with new-ish hardware, eg microcontrollers to get at least
a square wave output on a pin or some "Hello, World!" on a UART.
Post by Josef Möllers
JSON parsing is the sort of thing you can easily do in C, but you
really don't want to do yourself, becuase it is a lot of fiddly code
to write, and everyone wants to solve exactly the same problem, which
is to query text formatted as JSON, usually for fields whose names and
types are known, but with facilities to list fields onthe fly.
Yes, that was exactly my idea when writing the program: why come up with
something new when there is existing code to build upon.

To answer jak's question:
This is code solely for myself to translate a json string I receive from
a Shelly Plug S into separate MQTT topics for the various items in the
string, most important "output", but also "voltage" and "apower".
Most of my handlers rely on the fact that a topic carries only a single
payload value, not a json object.

Thanks to all for helping!

Josef
Mikko
2024-05-16 16:27:00 UTC
Reply
Permalink
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1,
"tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
--
Mikko
Josef Möllers
2024-05-16 18:01:15 UTC
Reply
Permalink
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.

What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.

But, as I wrote in another posting, I have switched to libjson-c5 and
the code works fine.

Thanks anyway for helping.

Josef
Scott Lurndal
2024-05-16 18:53:32 UTC
Reply
Permalink
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex). No parser needed.
bart
2024-05-16 19:21:28 UTC
Reply
Permalink
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex). No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.

A bit contradictory?

Clearly parsing IS needed, so either you've used LEX from code that acts
as a parser, or it does more than just recognise tokens.
Scott Lurndal
2024-05-16 20:54:37 UTC
Reply
Permalink
Post by bart
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex). No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.
A bit contradictory?
Clearly parsing IS needed, so either you've used LEX from code that acts
as a parser, or it does more than just recognise tokens.
No, it's just rearranging the tokens as they are
recognized into a RPN stack that can subsequently be evaluated.
Malcolm McLean
2024-05-17 04:24:31 UTC
Reply
Permalink
Post by bart
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex).  No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.
A bit contradictory?
Clearly parsing IS needed, so either you've used LEX from code that acts
as a parser, or it does more than just recognise tokens.
Yes.

I took Ben's advice and completely rewrote my XML parser wirh a formal
lexer and recursive descent grammar. It was good idea, even though XML
is simple enough to get away with a ad hoc approach. Similarly, with
JSON. once you've lexed into identifiers, values, and curly and square
btackets, the grammar is so simple that you almost don't notice that you
are writing a parser.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
bart
2024-05-17 14:13:07 UTC
Reply
Permalink
Post by Malcolm McLean
Post by bart
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex).  No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.
A bit contradictory?
Clearly parsing IS needed, so either you've used LEX from code that
acts as a parser, or it does more than just recognise tokens.
Yes.
I took Ben's advice and completely rewrote my XML parser wirh a formal
lexer and recursive descent grammar. It was good idea, even though XML
is simple enough to get away with a ad hoc approach.
'Deceptively simple' might be more apt. These schemes are always a bit
more involved than they look. Although this is also to do with the
subsequent representation than just parsing.

Isn't this the one with optional open/close tags, and optional
attributes when those tags are provided?

Fortunately I've haven't had to work with XML for years. It is anyway an
ugly format and pretty much unreadable even for humans, with the actual
data barely discernable amidst all the tag clutter. Here's a real example:

<Declarant>
<Declarant_code>00013464-00002735</Declarant_code>
<Declarant_name>
....
<Declarant_representative>
<null />
</Declarant_representative>
<Reference>
<Year>2017</Year>
<Number>001A</Number>
</Reference>
</Declarant>
Lawrence D'Oliveiro
2024-05-25 02:53:19 UTC
Reply
Permalink
Post by bart
Fortunately I've haven't had to work with XML for years. It is anyway an
ugly format and pretty much unreadable even for humans, with the actual
<Declarant>
<Declarant_code>00013464-00002735</Declarant_code>
<Declarant_name>
....
<Declarant_representative>
<null />
</Declarant_representative>
<Reference>
<Year>2017</Year> <Number>001A</Number>
</Reference>
</Declarant>
I’ve handled worse. How about this (after fixing that missing closing
tag):

<Declarant>
<Declarant_code>
00013464-00002735
</Declarant_code>
<Declarant_name>
....
</Declarant_name>
<Declarant_representative>
<null/>
</Declarant_representative>
<Reference>
<Year>
2017
</Year>
<Number>
001A
</Number>
</Reference>
</Declarant>

Courtesy of <https://bitbucket.org/ldo17/xmlpretty/>.
jak
2024-05-17 16:39:01 UTC
Reply
Permalink
Post by Malcolm McLean
Post by bart
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1,
"tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex).  No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.
A bit contradictory?
Clearly parsing IS needed, so either you've used LEX from code that
acts as a parser, or it does more than just recognise tokens.
Yes.
I took Ben's advice and completely rewrote my XML parser wirh a formal
lexer and recursive descent grammar. It was good idea, even though XML
is simple enough to get away with a ad hoc approach. Similarly, with
JSON. once you've lexed into identifiers, values, and curly and square
btackets, the grammar is so simple that you almost don't notice that you
are writing a parser.
  --
https://www.lulu.com/spotlight/bgy1mm
Don't you think it is overkill to use a compiler of compilers to parse a
Json? The function I wrote for parse the Json is just 140 code lines in
C, while I found examples with much more bulky with lex & yacc.
Malcolm McLean
2024-05-17 17:07:09 UTC
Reply
Permalink
Post by jak
Post by Malcolm McLean
Post by bart
Post by Scott Lurndal
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a
"Shelly Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1, "tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a "scanner",
it can only recognize tokens, eg the brackets, quotes, identifiers etc,
but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too
heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
I've done full expression parsing and subsequent evaluation with
lex (or flex).  No parser needed.
You've done 'full expression PARSING', then you say 'No PARSER needed'.
A bit contradictory?
Clearly parsing IS needed, so either you've used LEX from code that
acts as a parser, or it does more than just recognise tokens.
Yes.
I took Ben's advice and completely rewrote my XML parser wirh a formal
lexer and recursive descent grammar. It was good idea, even though XML
is simple enough to get away with a ad hoc approach. Similarly, with
JSON. once you've lexed into identifiers, values, and curly and square
btackets, the grammar is so simple that you almost don't notice that
you are writing a parser.
   --
https://www.lulu.com/spotlight/bgy1mm
Don't you think it is overkill to use a compiler of compilers to parse a
Json?  The function I wrote for parse the Json is just 140 code lines in
C, while I found examples with much more bulky with lex & yacc.
Yes. I did't use lex and yacc.

I wrote a lexer in top of a strem of characters, the I coded the
recursive decent parser by hand, with each rule a separate function.

Here's the new code
https://github.com/MalcolmMcLean/xmltocsv

And here's the old code it replaced
https://github.com/MalcolmMcLean/babyxrc/blob/master/src/xmlparser.c
https://github.com/MalcolmMcLean/babyxrc/blob/master/src/xmlparser.h
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
Mikko
2024-05-18 13:21:52 UTC
Reply
Permalink
Post by Josef Möllers
Post by Mikko
Post by Josef Möllers
Hi all,
I am trying to parse a json string received through MQTT from a "Shelly
Plug S", e.g.
{"id":0, "source":"button", "output":true, "apower":0.0,
"voltage":237.9, "current":0.000,
"aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1715172119},"temperature":{"tC":41.1,
"tF":106.0}}
I am trying to use libjson-glib but I can't figure out what to use as
the first argument to json_gobject_from_data()!
I am also looking at libjson-c but cannot find any examples that could
guide me.
Thanks in advance,
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Sorry to be nitpicking, but lex is a "lexical analyzer" aka a
"scanner", it can only recognize tokens, eg the brackets, quotes,
identifiers etc, but not structures.
"yacc" (or its "bison" equivalent" would be a parser but definitely too heavy.
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
As YACC is too heavy I would use only LEX and do with C what LEX alone
cannot do. I havn't often done so but when I did I got what I wanted.
For JSON parsing a stack of partly parsed objets and arrays is sufficient
in addition to what LEX gives, so the C code can be kept simple. LEX
can put C fragment to the right place in the final code.

If a generic parser is not needed but only a parser with specific structures
known in advance the result of parsing can be delivered in a form that is
easier to use than a result from a generic JSON parser. In addition, the
LEXä+C code can simpler in some sense although more complicated in another
sense.
Post by Josef Möllers
But, as I wrote in another posting, I have switched to libjson-c5 and
the code works fine.
That the code works fine is the most important aspect.
--
Mikko
Josef Möllers
2024-05-27 11:51:53 UTC
Reply
Permalink
[...]
Post by Mikko
Post by Josef Möllers
What one could do would be to use LEX to recognize the tokens and the
write a recursive descent parser in plain C.
As YACC is too heavy I would use only LEX and do with C what LEX alone
cannot do.
That is exactly what I wrote.

[...]
Post by Mikko
Post by Josef Möllers
But, as I wrote in another posting, I have switched to libjson-c5 and
the code works fine.
That the code works fine is the most important aspect.
In my 40+ years of experience in IT/programming (I graduated 1981 from a
Dutch polytechnic "HIO" in Computer Science and have retired in 2022) I
have learnt that "works fine" is only part of the work. Maintainability
should be added as well. Even if it is code written for one's personal
use only, it may need some work later and then it's crucial to have it
maintainable.

But maybe you think so too,

Josef
Vir Campestris
2024-05-27 20:18:47 UTC
Reply
Permalink
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981 from a
Dutch polytechnic "HIO" in Computer Science and have retired in 2022) I
have learnt that "works fine" is only part of the work. Maintainability
should be added as well. Even if it is code written for one's personal
use only, it may need some work later and then it's crucial to have it
maintainable.
But maybe you think so too,
I agree completely.

I learned my lesson a a student with a personal project which I left for
6 months. When I came back to it I had to comment it before I could
carry on.

(and I'm even older than you, although not by much)

Andy
Josef Möllers
2024-05-28 10:33:02 UTC
Reply
Permalink
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981 from
a Dutch polytechnic "HIO" in Computer Science and have retired in
2022) I have learnt that "works fine" is only part of the work.
Maintainability should be added as well. Even if it is code written
for one's personal use only, it may need some work later and then it's
crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I left for
6 months. When I came back to it I had to comment it before I could
carry on.
You will never have enough comments, even when you consider this rule ;-)
Post by Vir Campestris
(and I'm even older than you, although not by much)
Let me challenge that: I'm 67 (born September 1956) ;-)

Josef
Michael S
2024-05-28 11:45:09 UTC
Reply
Permalink
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
Josef Möllers
2024-05-28 18:15:15 UTC
Reply
Permalink
Post by Michael S
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
True. My (the obvious?) rule of thumb is that whenever I have to think
about wtf Iwas thinking, then I add a comment describing why I did what
I did. This often happens when I re-read the code before saving it, but
sometimes shortly after that. When I have to think about why I did what
I did much later, then it's obviously too late.

A remedy against too much/many comment(s) is to write some thorough
documentation.

And, also, don't forget boilerplates for functions!

Josef
Malcolm McLean
2024-05-29 16:14:41 UTC
Reply
Permalink
Post by Michael S
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
People have done some research and found that comments tend to make code
harder to understand and maintain. Unfortunately as always I can only
vaguely remeember reading this somewhere and can't provide any sort of
reference.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
David Brown
2024-05-29 17:23:06 UTC
Reply
Permalink
Post by Malcolm McLean
Post by Michael S
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
People have done some research and found that comments tend to make code
harder to understand and maintain. Unfortunately as always I can only
vaguely remeember reading this somewhere and can't provide any sort of
reference.
Too many comments make code hard to read - too much of a good thing is
always bad. (That's what "too much" means.)

The key, IMHO, is not to use comments when the same thing can be
expressed in code. And don't use comments to repeat things that are
obvious in the code. If a variable needs a comment to say what it is,
it's a sign that the variable should have a better name. It's better to
use an assert (static if possible) than to write a comment about the
state of variables. It's better to write clearer code than write
comments about what the code is doing.

Bad comments like those are a pain for maintenance - you often have to
double up the effort when fixing things, because you need to change the
code /and/ the comment. In practice, comments often get out of sync and
then they are worse than useless.

Of course you want comments to talk about /why/ code is doing what it is
doing, since that is usually not possible to express in the code itself.
Malcolm McLean
2024-05-29 22:32:01 UTC
Reply
Permalink
Post by David Brown
Post by Malcolm McLean
Post by Michael S
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
People have done some research and found that comments tend to make
code harder to understand and maintain. Unfortunately as always I can
only vaguely remeember reading this somewhere and can't provide any
sort of reference.
Too many comments make code hard to read - too much of a good thing is
always bad.  (That's what "too much" means.)
The key, IMHO, is not to use comments when the same thing can be
expressed in code.  And don't use comments to repeat things that are
obvious in the code.  If a variable needs a comment to say what it is,
it's a sign that the variable should have a better name.  It's better to
use an assert (static if possible) than to write a comment about the
state of variables.  It's better to write clearer code than write
comments about what the code is doing.
Bad comments like those are a pain for maintenance - you often have to
double up the effort when fixing things, because you need to change the
code /and/ the comment.  In practice, comments often get out of sync and
then they are worse than useless.
Of course you want comments to talk about /why/ code is doing what it is
doing, since that is usually not possible to express in the code itself.
Currently I'm writing a lot of little test programs to show off my XML
parser. They are of course also intended to be used as templates for
anybody who wants to use it. But I haven't decided on a commenting
policy. The comments are inconsistent and a bit of a mess.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
Tim Rentsch
2024-05-31 02:23:49 UTC
Reply
Permalink
Post by Michael S
On Tue, 28 May 2024 12:33:02 +0200
Post by Josef Möllers
Post by Vir Campestris
Post by Josef Möllers
In my 40+ years of experience in IT/programming (I graduated 1981
from a Dutch polytechnic "HIO" in Computer Science and have
retired in 2022) I have learnt that "works fine" is only part of
the work. Maintainability should be added as well. Even if it is
code written for one's personal use only, it may need some work
later and then it's crucial to have it maintainable.
But maybe you think so too,
I agree completely.
I learned my lesson a a student with a personal project which I
left for 6 months. When I came back to it I had to comment it
before I could carry on.
You will never have enough comments, even when you consider this rule ;-)
Probably true.
And despite that you can very easily have too much (or too many?)
comments.
I realize the original remark was tongue in cheek.. still, I have
some serious responses to offer.

A more important question then whether there are enough comments is
what kind of comments are given. It's easy to have too many of the
wrong kinds of comments. Most of the code I have looked at that is
is commented tends to have more of the unhelpful kinds of comments
than it has more helpful kinds of comments.

Whether there are enough comments (yes I think there can be enough
comments) also depends on how the code is written. Some code can
get by with very few comments. At the other end of the spectrum,
some code is so awful that adding comments will probably make things
worse rather than better. In cases like that, re-writing the code
has a higher ROI than it does to write more comments.
Lawrence D'Oliveiro
2024-08-05 02:55:13 UTC
Reply
Permalink
Post by Michael S
And despite that you can very easily have too much (or too many?)
comments.
My rule with comments is that they should explain *why* the code is doing
what it’s doing, not simply repeat *what* the code is doing (which would
be obvious to anyone who knows the language, APIs etc).

E.g.

const ssize_t allocation_step = 10;
/* something convenient to reduce nr of tuple resize operations */
...
PyTuple_SET_ITEM(factorelt, 0, factorobj);
PyTuple_SET_ITEM(factorelt, 1, powerobj);
factorobj = powerobj = NULL; /* ownership has passed to factorelt */
if (nr_used == nr_allocated)
{
/* need more room in result tuple */
nr_allocated += allocation_step;
if (_PyTuple_Resize(&tempresult, nr_allocated) != 0)
break;
} /*if*/

Vir Campestris
2024-05-30 16:49:50 UTC
Reply
Permalink
Let me challenge that: I'm 67 (born September 1956) 😉
I think we've given enough personal information in a public group (am I
paranoid enough_?) but we're not far apart.

Andy
Lawrence D'Oliveiro
2024-05-23 03:40:39 UTC
Reply
Permalink
Post by Mikko
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Fun fact: hand-coded lexical analyzers tend to be faster than table-driven
ones like lex.
Scott Lurndal
2024-05-23 17:14:16 UTC
Reply
Permalink
Post by Lawrence D'Oliveiro
Post by Mikko
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Fun fact: hand-coded lexical analyzers tend to be faster than table-driven
ones like lex.
You have a warped sense of fun.

Tend to be? Any cites?

And how often does the (I suspect very marginal) performance benefits
of a hand-coded lexical analyzer surpass the development/testing cost of using
a well-tested lexical analyzer generated by lex or flex, both of which
are rather efficient?
Malcolm McLean
2024-05-23 19:57:45 UTC
Reply
Permalink
Post by Scott Lurndal
Post by Lawrence D'Oliveiro
Post by Mikko
Sometimes it is easier to make a parser than to use an existing one.
In this case I might try LEX.
Fun fact: hand-coded lexical analyzers tend to be faster than table-driven
ones like lex.
You have a warped sense of fun.
Tend to be? Any cites?
And how often does the (I suspect very marginal) performance benefits
of a hand-coded lexical analyzer surpass the development/testing cost of using
a well-tested lexical analyzer generated by lex or flex, both of which
are rather efficient?
I"m currently adding xpath suppport to my XML document model.
The code is here

https://github.com/MalcolmMcLean/babyxrc/blob/topic/dataimport/src/xpath.c

(Baby X resource compilier, branch topic/dataimport, file xpath.c.)

It has a hand-coded lexical analyser. Now could the code be made better
by moving to lex / flex?

I'm working on the code so it is currently in flux. But github stores a
list of changes.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm
Loading...