Full Disclosure: I am one of the primary creators of JSON-LD, lead editor on the JSON-LD 1.0 specification, and chair of the JSON-LD Community Group. This is an opinionated piece about JSON-LD. A number of people in this space don’t agree with my viewpoints. My statements should not be construed as official statements from the JSON-LD Community Group, W3C, or Digital Bazaar (my company) in any way, shape, or form. I’m pretty harsh about the technologies covered in this article and want to be very clear that I’m attacking the technologies, not the people that created them. I think most of the people that created and promote them are swell and I like them a lot, save for a few misguided souls, who are loveable and consistently wrong.
JSON-LD became an official Web Standard last week. This is after exactly 100 teleconferences typically lasting an hour and a half, fully transparent with text minutes and recorded audio for every call. There were 218+ issues addressed, 2,000+ source code commits, and 3,102+ emails that went through the JSON-LD Community Group. The journey was a fairly smooth one with only a few jarring bumps along the road. The specification is already deployed in production by companies like Google, the BBC, HealthData.gov, Yandex, Yahoo!, and Microsoft. There is a quickly growing list of other companies that are incorporating JSON-LD. We’re off to a good start.
In the previous blog post, I detailed the key people that brought JSON-LD to where it is today and gave a rough timeline of the creation of JSON-LD. In this post I’m going to outline the key decisions we made that made JSON-LD stand out from the rest of the technologies in this space.
I’ve heard many people say that JSON-LD is primarily about the Semantic Web, but I disagree, it’s not about that at all. JSON-LD was created for Web Developers that are working with data that is important to other people and must interoperate across the Web. The Semantic Web was near the bottom of my list of “things to care about” when working on JSON-LD, and anyone that tells you otherwise is wrong. 😛
TL;DR: The desire for better Web APIs is what motivated the creation of JSON-LD, not the Semantic Web. If you want to make the Semantic Web a reality, stop making the case for it and spend your time doing something more useful, like actually making machines smarter or helping people publish data in a way that’s useful to them.
Why JSON-LD?
If you don’t know what JSON-LD is and you want to find out why it is useful, check out this video on Linked Data and this one on an Introduction to JSON-LD. The rest of this post outlines the things that make JSON-LD different from the traditional Semantic Web / Linked Data stack of technologies and why we decided to design it the way that we did.
Decision 1: Decrypt the Cryptic
Many W3C specifications are so cryptic that they require the sacrifice of your sanity and a secret W3C decoder ring to read. I never understood why these documents were so difficult to read, and after years of study on the matter, I think I found the answer. It turns out that most specification editors are just crap at writing.
It’s not like many of the things that are in most W3C specifications are complicated, it’s just that the editor is bad at explaining them to non-implementers, which are most of the web developers that end up reading these specification documents. This approach is often defended by raising the point that readability of the specification by non-implementers is viewed as secondary to its technical accuracy for implementers. The audience is the implementer, and you are expected to cater to them. To counter that point, though, we all know that technical accuracy is a bad excuse for crap writing. You can write something that is easy to understand and technically accurate, it just takes more effort to do that. Knowing your audience helps.
We tried our best to eliminate complex techno-babble from the JSON-LD specification. I made it a point to not mention RDF at all in the JSON-LD 1.0 specification because you didn’t need to go off and read about it to understand what was going on in JSON-LD. There was tremendous push back on this point, which I’ll go into later, but the point is that we wanted to communicate at a more conversational level than typical Internet and Web specifications because being pedantic too early in the spec sets the wrong tone.
It didn’t always work, but it certainly did set the tone we wanted for the community, which was that this Linked Data stuff didn’t have to seem so damn complicated. The JSON-LD 1.0 specification starts out by primarily using examples to introduce key concepts. It starts at basics, assuming that the audience is a web developer with modest training, and builds its way up slowly into more advanced topics. The first 70% of the specification contains barely any normative/conformance language, but after reading it, you know what JSON-LD can do. You can look at the section on the JSON-LD Context to get an idea of what this looks like in practice.
This approach wasn’t a wild success. Reading sections of the specification that have undergone feedback from more nitpicky readers still make me cringe because ease of understanding has been sacrificed at the alter of pedantic technical accuracy. However, I don’t feel embarrassed to point web developers to a section of the specification when they ask for an introduction to a particular feature of JSON-LD. There are not many specifications where you can do that.
Decision 2: Radical Transparency
One of the things that has always bothered me about W3C Working Groups is that you have to either be an expert to participate, or you have to be a member of the W3C, which can cost a non-trivial amount of money. This results in your typical web developer being able to comment on a specification, but not really having the ability to influence a Working Group decision with a vote. It also hobbles the standards-making community because the barrier to entry is perceived as impossibly high. Don’t get me wrong, the W3C staff does as much as they can to drive inclusion and they do a damn good job at it, but that doesn’t stop some of their member companies from being total dicks behind closed door sessions.
The W3C is a consortium of mostly for-profit companies and they have things they care about like market share, quarterly profits, and drowning goats (kidding!)… except for GoatCoats.com, anyone can join as long as you pay the membership dues! My point is that because there is a lack of transparency at times, it makes even the best Working Group less responsive to the general public, and that harms the public good. These closed door rules are there so that large companies can say certain things without triggering a lawsuit, which is sometimes used for good but typically results in companies being jerks and nobody finding out about it.
So, in 2010 we kicked off the JSON-LD work by making it radically open and we fought for that openness every step of the way. Anyone can join the group, anyone can vote on decisions, anyone can join the teleconferences, there are no closed door sessions, and we record the audio of every meeting. We successfully kept the technical work on the specification this open from the beginning to the release of JSON-LD 1.0 web standard a week ago. People came and went from the group over the years, but anyone could participate at any level and that was probably the thing I’m most proud of regarding the process that was used to create JSON-LD. Had we not have been this open, Markus Lanthaler may have never gone from being a gifted student in Italy to editor of the JSON-LD API specification and now leader of the Hypermedia Driven Web APIs community. We also may never have had the community backing to do some of the things we did in JSON-LD, like kicking RDF in the nuts.
Decision 3: Kick RDF in the Nuts
RDF is a shitty data model. It doesn’t have native support for lists. LISTS for fuck’s sake! The key data structure that’s used by almost every programmer on this planet and RDF starts out by giving developers a big fat middle finger in that area. Blank nodes are an abomination that we need, but they are applied inconsistently in the RDF data model (you can use them in some places, but not others). When we started with JSON-LD, RDF didn’t have native graph support either. For all the “RDF data model is elegant” arguments we’ve seen over the past decade, there are just as many reasons to kick it to the curb. This is exactly what we did when we created JSON-LD, and that really pissed off a number of people that had been working on RDF for over a decade.
I personally wanted JSON-LD to be compatible with RDF, but that’s about it. You could convert JSON-LD to and from RDF and get something useful, but JSON-LD had a more sane data model where lists were a first-class construct, you had generalized graphs, and you could use JSON-LD using a simple library and standard JSON tooling. To put that in perspective, to work with RDF you typically needed a quad store, a SPARQL engine, and some hefty libraries. Your standard web developer has no interest in that toolchain because it adds more complexity to the solution than is necessary.
So screw it, we thought, let’s create a graph data model that looks and feels like JSON, RDF and the Semantic Web be damned. That’s exactly what we did and it was working out pretty well until…
Decision 4: Work with the RDF Working Group. Whut?!
Around mid-2012, the JSON-LD stuff was going pretty well and the newly chartered RDF Working Group was going to start work on RDF 1.1. One of the work items was a serialization of RDF for JSON. The lead solutions for RDF in JSON were things like the aptly named RDF/JSON and JTriples, both of which would look incredibly foreign to web developers and continue the narrative that the Semantic Web community creates esoteric solutions to non-problems. The biggest problem being that many of the participants in the RDF Working Group at the time didn’t understand JSON.
The JSON-LD group decided to weigh in on the topic by pointing the RDF WG to JSON-LD as an example of what was needed to convince people that this whole Linked Data thing could be useful to web developers. I remember the discussions getting very heated over multiple months, and at times, thinking that the worst thing we could do to JSON-LD was to hand it over to the RDF Working Group for standardization.
It is at that point that David Wood, one of the chairs of the RDF Working Group, phoned me up to try and convince me that it would be a good idea to standardize the work through the RDF WG. I was very skeptical because there were people in the RDF Working Group who drove some of thinking that I had grown to see as toxic to the whole Linked Data / Semantic Web movement. I trusted Dave Wood, though. I had never seen him get religiously zealous about RDF like some of the others in the group and he seemed to be convinced that we could get JSON-LD through without ruining it. To Dave’s credit, he was mostly right. 🙂
Decision 5: Hate the Semantic Web
It’s not that the RDF Working Group was populated by people that are incompetent, or that I didn’t personally like. I’ve worked with many of them for years, and most of them are very intelligent, capable, gifted people. The problem with getting a room full of smart people together is that the group’s world view gets skewed. There are many reasons that a working group filled with experts don’t consistently produce great results. For example, many of the participants can be humble about their knowledge so they tend to think that a good chunk of the people that will be using their technology will be just as enlightened. Bad feature ideas can be argued for months and rationalized because smart people, lacking any sort of compelling real world data, are great at debating and rationalizing bad decisions.
I don’t want people to get the impression that there was or is any sort of animosity in the Linked Data / Semantic Web community because, as far as I can tell, there isn’t. Everyone wants to see this stuff succeed and we all have our reasons and approaches.
That said, after 7+ years of being involved with Semantic Web / Linked Data, our company has never had a need for a quad store, RDF/XML, N3, NTriples, TURTLE, or SPARQL. When you chair standards groups that kick out “Semantic Web” standards, but even your company can’t stomach the technologies involved, something is wrong. That’s why my personal approach with JSON-LD just happened to be burning most of the Semantic Web technology stack (TURTLE/SPARQL/Quad Stores) to the ground and starting over. It’s not a strategy that works for everyone, but it’s the only one that worked for us, and the only way we could think of jarring the more traditional Semantic Web community out of its complacency.
I hate the narrative of the Semantic Web because the focus has been on the wrong set of things for a long time. That community, who I have been consciously distancing myself from for a few years now, is schizophrenic in its direction. Precious time is spent in groups discussing how we can query all this Big Data that is sure to be published via RDF instead of figuring out a way of making it easy to publish that data on the Web by leveraging common practices in use today. Too much time is spent assuming a future that’s not going to unfold in the way that we expect it to. That’s not to say that TURTLE, SPARQL, and Quad stores don’t have their place, but I always struggle to point to a typical startup that has decided to base their product line on that technology (versus ones that choose MongoDB and JSON on a regular basis).
I like JSON-LD because it’s based on technology that most web developers use today. It helps people solve interesting distributed problems without buying into any grand vision. It helps you get to the “adjacent possible” instead of having to wait for a mirage to solidify.
Decision 6: Believe in Consensus
All this said, you can’t hope to achieve anything by standing on idealism alone and I do admit that some of what I say above is idealistic. At some point you have to deal with reality, and that reality is that there are just as many things that the RDF and Semantic Web initiative got right as it got wrong. The RDF data model is shitty, but because of the gauntlet thrown down by JSON-LD and a number of like-minded proponents in the RDF Working Group, the RDF Data Model was extended in a way that made it compatible with JSON-LD. As a result, the gap between the RDF model and the JSON-LD model narrowed to the point that it became acceptable to more-or-less base JSON-LD off of the RDF model. It took months to do the alignment, but it was consensus at its best. Nobody was happy with the result, but we could all live with it.
To this day I assert that we could rip the data model section out of the JSON-LD specification and it wouldn’t really affect the people using JSON-LD in any significant way. That’s consensus for you. The section is in there because other people wanted it in there and because the people that didn’t want it in there could very well have turned out to be wrong. That’s really the beauty of the W3C and IETF process. It allows people that have seemingly opposite world views to create specifications that are capable of supporting both world views in awkward but acceptable ways.
JSON-LD is a product of consensus. Nobody agrees on everything in there, but it all sticks together pretty well. There being a consensus on consensus is what makes the W3C, IETF, and thus the Web and the Internet work. Through all of the fits and starts, permathreads, pedantry, idealism, and deadlock, the way it brings people together to build this thing we call the Web is beautiful thing.
Postscript
I’d like to thank the W3C staff that were involved in getting JSON-LD to offical Web standard status (and the staff, in general, for being really fantastic people). Specifically, Ivan Herman for simultaneously pointing out all of the problems that lay in the road ahead while also providing ways to deal with each one as we came upon them. Sandro Hawke for pushing back against JSON-LD, but always offering suggestions about how we could move forward. I actually think he may have ended up liking JSON-LD in the end :). Doug Schepers and Ian Jacobs for fighting for W3C Community Groups, without which JSON-LD would not have been able to plead the case for Web developers. The systems team and publishing team who are unknown to most of you, but work tirelessly to ensure that everything continues to operate, be published, and improve at W3C.
From the RDF Working group, the chairs (David Wood and Guus Schreiber), for giving JSON-LD a chance and ensuring that it got a fair shake. Richard Cyganiak for pushing us to get rid of microsyntaxes and working with us to try and align JSON-LD with RDF. Kingsley Idehen for being the first external implementer of JSON-LD after we had just finished scribbling the first design down on paper and tirelessly dogfooding what he preaches. Nobody does it better. The rest of the RDF Working Group members without which JSON-LD would have escaped unscathed from your influence, making my life a hell of a lot easier, but leaving JSON-LD and the people that use it in a worse situation had you not been involved.
Are there any non-video links you would recommend for those wanting to know more about json-ld?
I’m sure you’re aware that for many technical folks, videos are the worst way to try and learn something; text rules!
thanks!
Yeah, start here: http://www.w3.org/TR/json-ld/#basic-concepts
You can play around w/ JSON-LD here: http://json-ld.org/playground/
More articles and presentations here: http://json-ld.org/learn.html
Actually reading this article is informative, as is google, of course. Still, any specific learning links you may have would probably be useful.
I would just like to say: Thank You !
JSON-LD is a beautiful standard. I am working on making the uniprot rdf data available in JSON-LD, and it is starting to look like something that a JS developer would actually use. Its still RDF, but at the same time its practical and usable JSON.
At the same time SPARQL construct returning results in JSON-LD is a really nice way to build quick data applications with just a quad store and some JS files!
Thank you!
Hey Jerven, thanks! And you’re welcome! That you’re using it on the Uniprot data set is awesome, let us know if we can help in any way. I have a personal interest in the stuff you’re working on: http://manu.sporny.org/2011/public-domain-genome/ http://manu.sporny.org/2011/pd-genome-implications/
Will probably take you up on that offer for helping.
Our RDF used quite a bit of reification for attaching provenance to statements, something that we will most likely replace with graphs in JSON-LD, but it does hurt a bit. And better ideas are welcome.
Just so that you’re aware, JSON-LD fully supports named graphs, and has a generalized RDF Graph Normalization algorithm. This allows you to make provenance statements quite easily and digitally sign those provenance statements at the graph level.
One issue with JSON-LD that seems not so pragmatic: http://fsteeg.tumblr.com/post/74217720933/one-issue-with-json-ld-that-seems-not-so-pragmatic
There are two issues that you outline in your blog post. The first is “how do you get back an object that is indexed by @id”? The second is how do you get back a property that is indexed by language.
Good questions, and there are straightforward answers to both. But first, some observations. It looks like the JSON-LD data you’re working with is not in an optimal form, meaning, it’s not very JSON-LD-y. We tell people to avoid using @graph when they can, and the example you point to is a great example of when you shouldn’t use @graph (there really is only one root object and all the other objects are children of that object). @graph is meant to be used when there are two objects that you want to express in the same document and they’re not connected to each other in any way. The data that should be returned to you should have http://lobid.org/organisation/DE-605 as the top-level object, and should embed _:t0, _:t1, #n02, and #n11 inside that object. Here’s what it should look like:
https://gist.github.com/msporny/8571005
You will also note that your second question is addressed in the data above. You can index language-specific values associated with a property by using a language map (see example 34): http://www.w3.org/TR/json-ld/#string-internationalization
The answer to your first question is a function that’s a few lines of JavaScript/Python/PHP/etc code, which would scan the incoming JSON-LD object and build an index using the @id of the object. As you can imagine, people wanted things to be indexed in a large variety of ways, and in almost every case, you can write a really simple function to do it. Adding the feature to the language would have overly complicated matters and we would’ve had to have /many/ types of indexed-by approaches (index by @id, index by @type, index by arbitrary property, etc.). That’s not to say that we might not add this into a future version of JSON-LD, but for now, it wasn’t clear if the tradeoff in feature complexity was worth it, and we did also know that it’s only a few lines of code to do the indexing if people really wanted it.
Thanks! Reply and context of my original post on Twitter: https://twitter.com/fsteeg/status/426294538463092736
Manu, I appreciate the kind words, but I was never involved in the JSON-LD work, and I’m a bystander at best in most Community Groups, and while I advocated for their formation, I was only peripherally involved in helping get them up and running. Ian Jacobs is the one who built them and keeps the lights on there, and he deserves exclusive praise for that, not me.
That said, there are many great W3C Community Groups that I do follow, and I think they are going to be fruitful, thanks to the passion of the developer community. The one I’m most excited about right now is the Open Annotation CG, which started off with 2 competing annotation data model specs and managed to fold them into one better spec with wide consensus and the start of adoption, in stark defiance of XKCD-927!
I’d personally like to see the transparency and accountability of W3C continue to increase, and we do have some plans there. I hope to have something interesting to show you in the next 6 months!
Thanks for the write-up. You’ve always got a refreshingly… er… candid viewpoint. 🙂
If you’re not with us then you must be against us. Burn the heretic!
You were the one that convinced me to form a Community Group around JSON-LD. You probably didn’t know that, but there you go… so I’m keeping you in the credits, you lovably humble Missourian Viking you.
Looking forward to hearing the big announcement 6 months from now.
The RDF abstraction is the reason that Google was able to very rapidly swap from Microdata-in-email to JSON-LD-in-email without having to re-work any of the associated schema.org vocabularies, i.e. https://developers.google.com/gmail/actions/reference/formats/json-ld – if schema.org was not defined on top of RDF’s data model, this would not have been nearly so easy. Is RDF the wonder-datamodel, suitable for representing all programmatic data structures? Absolutely not. But in a world outgrowing the idea that we’ll all some day settle on XML, or JSON, or HTML5++, or [data flavour of the week], RDF’s simple-minded classes and properties graph abstraction does find a niche. JSON-LD is an important piece of the ‘data on the Web’ puzzle, many thanks for all your hard work getting this spec done.
As I’m sure you remember, Microdata wasn’t based on RDF at all. Representing Microdata as RDF was an implementation decision that Google made, against the author of Microdata’s “no RDF” intent. Even if we had taken that path with JSON-LD, I assume the same thing would’ve happened; a group of people at Google would have found a mapping from JSON-LD to RDF. We just did it preemptively since we saw that coming, but the RDF WG wasn’t happy with just a mapping to/from RDF, they wanted JSON-LD to be completely based on top of RDF, which led to this section in the spec: http://www.w3.org/TR/json-ld/#relationship-to-rdf
The Microdata RDF specification required a group of people to come in behind Microdata and establish how it mapped to RDF. JSON-LD started out very much in the same way, the data model was different from RDF (not by a lot, but in a way that was significant and that mattered). My point wasn’t that RDF’s data model isn’t useful, it’s that people tend to overemphasize its usefulness over other graph-based data structures (like the one that JSON-LD started out with). In addition, the RDF community has historically gotten in the way of /improving/ the RDF data model to the point that it could do things like represent a JSON document (we hit quite a bit of this friction during the development of JSON-LD). The RDF community has also pounced on nascent graph-based technologies claiming that they’re effectively RDF (which they may have been, but preemptively stomping on a nascent technology doesn’t allow it to develop along a natural path and see how it evolves. It may evolve into RDF, it may go a different route).
Before RDF 1.1, and even now, non-generalized RDF can’t represent a JSON document and provide a round-trip between RDF and JSON. Some may say that’s not important, but if you’re going to claim that RDF is a generalized data format, which some proponents of RDF do, then you should be able to do simple things that most developers are used to doing on a frequent basis (like losslessly represent a JSON document). It took 10+ years to get from RDF 1.0 to RDF 1.1 (and it was an incremental change). If we’re going to make RDF useful to the general programming population, we have to iterate faster and we can’t let history (ie: “This is the formal RDF, and it’s set in stone!”) get in the way of improving it.
So, while I agree with what you’re saying to a degree, I also think that there are practical ways of improving RDF’s data model. The thing that’s getting in the way of that is the RDF community, even though many have the best of intentions when arguing against changes to the RDF data model and other RDF-like technologies.
While we’re thanking each other, thank you for your work at Google on schema.org and other Linked Data-related technologies. It was always difficult engaging Google on matters such as these before you came along, so your outreach to the general Linked Data community is greatly appreciated. 🙂
Hi Manu,
With regard to “ I assert that we could rip the data model section out of the JSON-LD specification and it wouldn’t really affect the people using JSON-LD�, where do you see the added value of JSON-LD then?
Are people really waiting to give their JSON properties and/or universal meaning by turning them into URIs?
And how much is this �universal meaning� worth when there is no underlying model?
Isn’t it rather content types/schemas/profiles that we need for interoperability—and that’s it?
Ruben
The added value of JSON-LD wouldn’t change at all if there were no data model section. Computer/Data scientists have this misguided notion that you need a data model for something to be useful. I believe this notion typically comes from the scientific desire to have things rigorously defined and predictable, which isn’t necessarily a bad thing, but it drives people toward nailing something down before it’s really necessary.
To answer your other questions, no, developers don’t want to transform their JSON properties to URIs just for the sake of doing it. They do want to be able to merge and combine data sources, though. You can have universal meaning without having a rigorously defined data model. For example, just making the JSON properties map to URIs and giving a JSON object and identifier of some kind is enough. You don’t need RDF. That’s all you need to give the object universal meaning, everything else is icing on the cake.
​> Precious time is spent… discussing how we can query all this Big Data… instead of figuring out a way of making it easy to publish that data on the Web…
This has been irking me for some time. All attention seems to be focussed on doing things with data, yet precious little on making it easy to publish data.
If you’ll permit me a shameless plug….
I’ve been tackling this problem with a project of my own, aimed at making it easy to publish open data from Google Spreadsheets. http://opendatapress.org
Great project Craig, hope you add support for JSON-LD at some point. 🙂
ManuSporny, do you think microsyntaxes are effectively dead? What’s the alternative–JSON everywhere?
This depends heavily on what your definition of “microsyntax” is. My definition is “a way of encoding data in a span of text that expresses metadata about part of the contents in the span”. For example, this is a microsyntax: “7^^”
Do I think all microsyntaxes are effectively dead? No. However, any solution that requires a domain specific language, which a microsyntax can be considered a subset of, to be learned is going to be more difficult for a developer than trying to eliminate the need for the microsyntax in the first place. Eliminating the need comes at a cost, which is that the language becomes less expressive. JSON-LD almost had microsyntaxes for data until we realized that it would be awkward for Web developers to mark up numbers like this: “7^^“, instead of just using what they were familiar with.
Thank you Manu, you have just confirmed my objections about W3C vision of Semantic Web. You may find a very interesting and highly relative debate of mine at http://goo.gl/KDvkTL. I am posting also regularly at the Semantic Web Research group at LinkedIn. In brief, I believe there is going to be a better solution than RDF and Linked Data. I call it R3DM and it has a solid foundation based on Aristotle’s sign theory. You may read about this and a comparison of several Variable-Value-Pair data models, I call them VVP, at http://goo.gl/EWFmQZ. The conclusion is that you have managed to turn them to the right direction that of a VVP.
I’ve read all the links you sent as well as the links in those links. I still don’t understand the differences between R3DM and RDF. What are the advantages of VVP and R3DM over RDF? They seem like two abstract data models that do the same thing to me, and that you want to create a new referencing scheme w/o specifying exactly what that scheme would look like is confusing to me.
Yes, I agree the way it looks R3DM is an abstract data model. It is not something concrete you can work with. Perhaps it will help to say that I am trying to develop something like a Google PaaS technology that combines Freebase and NDB Datastore. In case you have studied these, you will notice that they are using VVP to assert facts into their database. The first one is similar to RDF but also different in many aspects, the second one is similar to JSON-LD but again different in many ways. In both cases what is important is how you construct complex objects, how you store or retrieve them, how you run queries against them, how you associate them, etc. This is also the core idea behind all the database models relational, graph, key-value, hierarchical, etc. Regarding to the new referencing scheme, I am trying to build it on top of the existing URL one. You are right to be confused, I am trying hard to build specifications and implement them at the same time in simple test cases.
Thanks!
Is there a list with JSON-LD libraries and implementations that I could point to when pitching JSON-LD?
Yeah, the list is on the landing page of the JSON-LD website:
http://json-ld.org/#developers
These videos might also help you pitch the technology:
http://www.youtube.com/watch?v=4x_xzT5eF5Q
http://www.youtube.com/watch?v=vioCbTo3C-4
My team has been developing applications with RDF and JSON for the last 15 months or so and we have come to different conclusions from yours. RDF has been extremely useful to us and has become more and more important over time. JSON-LD has not turned out to be helpful, but a few ideas stolen from JSON-LD have been important. RDF/JSON has been useful. I wrote up a rather long account of our experiences here if you are interested: http://tinyurl.com/kpk5zjy
Hi Manu, I remember JSON-LD from my W3C GLD days when I did a review on it. I like JSON-LD even though I never had to use it thus far. For the past 2 years, however, we’ve been developing stuff using RDF without running on too much trouble, actually no trouble at all. It’s just a matter of perspective, because in data modelling RDF is very useful in putting things into a more comprehensible conceptual context. And my personal opinion is that it is great in covering the gap between conceptualization and serialization/implementation. So based on personal experience I really do believe that RDF plays an important role there. Also, about the LIST support part, one could argue that it’s schema-less so there is limited native support whatsoever. At the instantiation and implementation level, however, one could provide all the support they desire, be it lists or whatnot.
Manu,
has it ever occured to you that “RDF is a shitty data model” because you use it with shitty old software development methods?
Yes, that thought had occurred to us. That said, we do use many of the RDF concepts in what we do. Don’t take my criticism of RDF as a suggestion that people shouldn’t use RDF. What I’m trying to convey is that RDF isn’t nearly as polished as some make it out to be. It needs some work, but it’s also better than many of the other data models out there when it comes to merging disparate data sources together.
Manu,
Great post. I am looking at implementing a complex command and control system for the Navy using JSON-LD. I have a question about the use of @id. From what I understand, the idea behind @id is that it provides a way to find/look up the item, or things the item links to directly. Given the complex nature of Navy networks with an abundance of firewalls, the internal servers may not know (in any ad-hoc fashion), the public addressable context in which to create @id’s. Also, it would not be performant to have internal processes go all the way out to the public servers to resolve @id’s. In a private network we could mess with DNS or set up proxys to deal with things like this, but that is much more difficult on DoD networks. Looking for some advice / wisdom on the proper way to use @id given that direct lookup is not always possible.
Hi Craig, you seem to understand the basic idea behind @id. In general, you can use @id if you want to give programs a hint to where they can find out more about a particular “thing”. Keep in mind that it’s not just @id that enables this, but any value in a JSON-LD document that is a URL. Every URL in a JSON-LD document is an opportunity to go out to the network and discover more about the resource pointed at by the URL.
Let me preface the next bit by saying that we do quite a bit of DoD contracting at Digital Bazaar and if you want to pursue a deeper conversation, we’d be happy to do it in a forum that is not a blog post comment thread. Just email us at support@digitalbazaar.com and we’d be happy to respond there. Keep in mind that we’re very busy and paid work always gets priority. 🙂
With respect to Navy networks (or any DoD network for that matter), your JSON-LD data doesn’t have to include links that go out to the public Internet. In most DoD scenarios, we suggest that you use identifiers that are completely internal to the Navy network (the Navy-reachable, but internal portions of the network). What those identifiers are ends up being up to which Command the identifiers are housed in. For example, identifiers for the Naval Meteorology and Oceanography Command would be housed on their servers while identifiers created and hosted by the Naval Air Systems Command would use their naming/identity scheme. Also note that there are ways to provide cryptographic proof of the caller via HTTP Signatures, Secure Messaging, or Identity Credentials:
https://web-payments.org/specs/source/http-signatures/
https://web-payments.org/specs/source/secure-messaging/
http://manu.sporny.org/2014/credential-based-login/
That is, it is possible to publish data on an internal Navy network, but make that data accessible by only those systems that need to know (all other requests for the data would be denied). As I said, we’ve been doing work like this for some time, so happy to pick up the conversation w/ you offline (as we can’t get into the details in a public forum).
Manu,
Congratulations to you and all of your contributing colleagues on this terrific accomplishment, which will benefit so many people and projects over time. -Mark
Manu,
Great post.
“We all know that technical accuracy is a bad excuse for crap writing”. Great quote here.
Some rants on specs and the community:
I think specifications should have versions for implementers, non-implementers and computers (machine-readable, logically tested, etc). As a developer I find harsh to read ambiguous RFCs or any technical work, specially for Computer Science, a science field that supposedly should be supported by Logic. This can lead us to segmented implementation, therefore, no standards to rely on, no interoperability and development nightmares! A quick example: GET requests (for queries in a REST API) with/without a payload.
Of course, the first contact with a spec should be the easiest one, i.e. to non-implementers (cool), then to implementers (precise), than a machine readable version of the spec itself (I have a dream).
However, as an observer of the Semantic Web community and developer-reseacher (in that order), these fights over Logic-Semantic-RDF guys versus Web-API-Linked-microdata people are extremely counter-productive. We got to find an agreement here to fulfill Linked Data (or Semantic Web’s) vision.
Can anyone point to some example of json-ld implementation inside the html code?
It#s still not clear to me where to place the code, if it is important at all
Thumbs up from the guy that complains about WebApp WG all the time.
Finally another one that has his own opinion and doesn’t do shit the same way as others just because they did so in the past.
DOM is the crappiest API designed by humanity, with HTML5 and strict mode in ES5 they finally had the chance to fix all the legacy shit – and nobody cared about it. Now they said it was a mistake doing so with the new modes, but guess what: It can’t be undone before you not introduce HTML6 (never will be, probably).
So many mistakes, so much legacy API that is designed for being buggy. Nobody wants to fix that and all in the JS world are on the semantic versioning trip – which is heavily wrong from the maintainer’s perspective. Imagine a security leak in jquery 0.8, never gets fixed, all websites stay buggy until the end of their server’s life.
I could start about problems with shipping code dependencies and not being able to ship minified “binaries” of your JS files across multiple platforms, hell – even architectures – but that would be a never ending story. Looking at you, package manager for JS. Nobody allows shipping any kind of shared library concept with security-fixes by maintainers among the web or native operating systems.
Looking forward to try out JSON-LD though 🙂
I’m dealing with RDF and SPARQL for a long time in OpenLink Software’s Virtuoso project and I can’t agree with you more when you describe the disadvantages of RDF. At the same time, I have W3C’s badge attached to a red W3C ribbon, so there should be some reason why I tolerated these disadvantages since 2005. And the reason is simple: RDF and SPARQL are not for human beings. This technology stack is primarily for robots, automated agents, provers… but not for us. That’s why RDF’s data model is oversimplified: it’s proven by decades of practice that more complicated models are not practical. P.H. Winston’s model is almost as simple as RDF and I like it, but an author of a silly bot needs something simpler, so the RDF has links to subjects but not to triples (and rdf:Statement), explicit Lisp-style CAR and CDR etc. At the querying side, SQL is an error-proof fortress if compared to vulnerable SPARQL, but nobody cares: robot will not debug its queries! Instead, robot should concatenate multiple fragments of a query into a page of terrible unreadable unexplainable crap and SPARQL endpoint should accept it and optimize from A to Z and try to execute it in a reasonable time. I’m maintaining Virtuoso’s SPARQL compiler and I see customer’s cases. Most of customers never write the queries they worry about; say, a human being can not write a 67Kb long query, esp. as a single line. Really, it may be practical to automate this bug reports, to let customer’s program file a case without annoying the customer himself. (Sometimes I feel that someone has forgotten to automate me, a proxy between robots that write queries and a local automated test suite.) Humans write programs that interact with each other in nearly unpredictable ways, it looks a bit crazy, and an average web designer will either use the very basic things or connect to some simple “facade” interface that will hide RDF/SPARQL complexities.
So thank you for JSON-LD, for keeping it suitable for simple use cases and thus for making it more compatible with legacy protein-based hardware.
Thank you ManuSporny for all of the fresh air. My introduction to linked data was to create a data structure for a push-pop link-list in beginning c programming. (After readingthat Tim Berners-Lee lost his original code for hyperlinks, I always wondered why they did not have a contest for some one to re-create the code.) I am numb from trying to learn RDF on the fly. I will certainly hunt down and read/watch every post/video you have made available. How soon will I get my decoder ring? THANK YOU.
RDF: https://www.w3.org/2010/02/rdfa/sources/rdf-api/
RDF provides a means to publish and exchange machine-readable data about resources on the Web
This enables web developers to process published metadata about for example a News article and combine it with additional information from other sources.
JSON-LD http://json-ld.org/
Linked Data empowers people that publish and use information on the Web. It is a way to create a network of standards-based, machine-readable data across Web sites. It allows an application to start at one piece of Linked Data, and follow embedded links to other pieces of Linked Data that are hosted on different sites across the Web.
JSON-LD is a lightweight Linked Data format. It is easy for humans to read and write. It is based on the already successful JSON format and provides a way to help JSON data interoperate at Web-scale. JSON-LD is an ideal data format for programming environments, REST Web services, and unstructured databases such as CouchDB and MongoDB.
spnote: this of course requires RFC 6570 and the infamous URI Templates. ? Salesforce.com ? seriously ?
I think I’ll stick with RDF.
trying to merge with RDF ==> huge mistake
there is simply NO ADDED VALUE whatsoever by employing RDF.
and every time a poor fool makes the misguided decision to go for this “W3C standard” and embed it in his product, then the RDF cult leaders announce it like a major breakthrough, that this is the beginning of the wave of adoption.
but the problem there is nothing that RDF (and all the crazy things built on top of it like OWL, OWL-S etc) brings to the table that pure xml or json could not do either, except making everything more cumbersome.
the point is: just reasoning about “triples” will not get you anywhere. but if your app needs to grasp more than “triples” than there is no point in modelling them..!!
RDF is not some kind of magic “DNA” that underpins all forms of AI, it’s just a stupid 20 year old idea that unfortunately got attention from the wrong people
Is there any json-ld real time examples.But this was very nice information
I don’t know what you mean by “real time”, but there is the JSON-LD Playground, where you can play around with examples and markup in realtime: https://json-ld.org/playground/
Can someone provide a 2 or 3 referrals of a Coder that can/ convert my Microdata Plug-In to JSON-LD?
Funny, I ran into this article today (2018) after sitting around my apartment while working from home and thinking to myself randomly, “hey, what the hell ever happened to RDF? man, that thing obviously went nowhere in a hurry.” So I googled “is RDF dead?” and here I am. Very informative indeed. I actually had no idea that JSON-LD–which I have implemented in production for various very large clients–had anything whatsoever to do with RDF in any form or fashion. I’d agree, you can rip that RDF Data Model section out. Actually, I wish someone would. RDF makes my brain hurt, after over a decade of re-reading the spec I still get cross-eyed just lightly skimming it.
I am so very glad to see you bring up 1) That developers were not the intended audience and the proliferation of the technology suffers for that and that 2) There are simply some terrible technical writers out there working on big projects. This is something of a passion point for me, and something I’ve been thinking about my entire career.
I’m entirely self-taught using free resources on the internet, and I remember what it was like especially early on to have to soldier through poorly worded and conceptualized documentation as best I could. And not just web development, low-level ASM, open-source hardware… So many projects that I theoretically believe in and would like to support, whether it’s a language or a framework or a library… but that I turned away from after maddening hours of bashing my head against a wall trying to understand some concept or another.
Many technical writers communicate in a style and from a background that is overly formalized, pedantically academic, and basically the verbal equivalent of a Code Obfuscation Contest.