Skip navigation

community

1 2 3 Previous Next 15956 Views 30 Replies Latest reply: Sep 14, 2011 5:31 AM by blalor RSS
Currently Being Moderated

Oct 13, 2010 4:45 PM

CadSoft EAGLE development team developing XML format for schematic, layout and parts libraries!

hello makers -

 

this is my first post here, i'm the senior editor at MAKE magazine and also the creative director at adafruit. i've been part of a growing movement called "open-source hardware" for the last 5 years or so. today we just got some great news from the CadSoft EAGLE development team about the their long term commitment to text based file formats in the world of electronics development. with their permission i am posting this here and on MAKE (as well as adafruit). it's extremely exciting for those of us who wanted an XML format for files and we're thrilled CadSoft EAGLE has said they're committed to it!

 

here's the overview of what we learned:

 

  • CadSoft EAGLE (our preferred schematic/layout software) has told us they are committed to a text based format in the long term.
  • The CadSoft EAGLE development team is developing a new format for schematic, layout and parts libraries that is XML, this means every object and line will be written out in text description.
  • You won't have to worry about binary file corruption, you will be able to hand-edit or generate schematics and layouts and of course the magic of version-control (such as github) will be easier than ever with real text 'diff's!
  • This will be a fully documented format and also of course a converter for old CadSoft EAGLE binary-type formats.

 

 

this is a great direction and i encourage you to support EAGLE as you decide which tools to use now and in the future. but that's not all, i've chatted with the element-14 folks and they're all set up for your feedback, if they're very interested in Maker voices their product development. so here's your chance folks!

 

cheers,

pt

Attributes

  • Personally I don't think that is a good idea. Xml itself introduces ambiguitiy, as does text. It is also language based (and large, slow to parse and all the regular issues). I would way rather they open the binary formats if they haven't already. The existing text formats (like the spice format) work fine and are way more readable than xml as is.

     

    I understand I may be a minority in this view : ).

  • Well seeing as binary can just be a cast I'd say it is slower - I agree it doesn't have to be 'too slow', but it often ends up that way.

     

    XML is more ambigous than a good documented text format imo - eg like spice, PS etc. For example you have tags vs attributes, or unicode vs ascii, etc. That can be doc'd, but as a user you have to remember if it is one, the other, or both. And that is 'if' it is doc'd properly. That said, it is binary now, not text - I think they use text for some of the config stuff (like hotkeys iirc) though, which is why I mentioned it...

     

    Most tools for xml formats have problems accepting files that are 'legal' as far as the schema once the format becomes a bit large. I've had lots of issues with that in both SVG and XAML, and of course HTML seems to be hard for everyone to get right on the edges (even harder for editors). With a binary format it tends to blow up or work, which imo is more desirable.

     

    As far as viewing, you can view it in Eagle. I think the format is mostly interesting to those who want to tool against it.

     

    The one major plus of a text format is version control. You could have a tool that converts to and from open binary <> test/xml for that, but I agree that is less optimal. So one large point for text formats there : ).

     

    Ah well, I'll take open xml is over closed binary any day, so I'm with you for sure... Just feel 'open binary' is never given fair consideration. Ironically I think its main strength is portability - not of the parser tool, but of the exact meaning intended.

  • Well, I for one will be very glad to see an XML file format implemented.  Back in my Cadence days I wrote a Java-based netlist parser app to convert between languages (mostly SPICE, Spectre, Verilog-AMS and VHDL-AMS) using XML as an intermediary format and XSL Transforms to interact with the source and generate the destination formats.

     

    Aside: I hate SPICE precisely because it is totally ambiguous, poorly documented and incompatible between all the permutations and combinations out there between EDA platforms.  At least the hierarchical nature of properly formatted XML is parsable independent of the content or even a formally defined schema.

     

    Aside 2: IMHO, if you have to use regular expressions to get at the atomic structures contained in an "XML" format like you do in SVG, it's not proper XML.  Things like coordinates should be accessible as discrete XML elements or attributes accessible through simple XSLT code.

     

    If size is a concern (which it was when dealing with GB-size netlists) you can use a generic zip algorithm to compress the text and serialize the I/O.  As a bonus you get encryption, authentication, error correction and all that other good serialization infrastructure for free.  It'll add a layer of I/O complexity for revision control, but if the RC supports generic zip-diffs, all text formats will benefit, not just XML.  You also get ascii vs unicode vs localization controls for free with most XML-based APIs, independent of whether the tools (don't) have it.

     

    Aside 3: Avoid the DOM-style XML readers that suck in the whole XML content before you can do anything with the content.  You're much better off using a SAX approach to create your own (sparse) data structures on the fly, with the added benefit that when data's corrupt you can more easily identify and work around where things are broken in the stream.

  • I'm super happy it is opening, and I think there is a real chance I will use it for something that will be useful to me... so double yaay! I should have led off with that, sorry : ).

     

    For the record, I'm not saying use text instead of xml - I'm saying binary has a lot of advantages you lose and xml has potential pitfalls. If you do everything right xml can be reasonable, but if most major uses of xml are 'doing it wrong' then maybe it needs to be a bit more explicit and enforceable. I can deal with it though, minor quibble. Eagle is an awesome dev tool - regardless of how it is serialized : ).

  • I'm all for openness and transparency in data files. If the data in your text file is large, though, then it can get pretty nasty for a human to parse and edit. I consitently deal with verilog netlists that are on the order of 100Mb in size for our ASICs at work. In fact, lots of files for my job are text (SPEF, netlist, DEF, LEF, LIB). We don't edit netlists by had because they are too large and complicated. We use EDA tools to manually manipulate netlists with languages like TCL (edit netlist disconnect, edit netlist connect, edit netlist create and_gate). I would recommend supporting the reading and writing of compressed XML files out of the gate. Why waste 100MB of disk space when gzip will compress it down to 2MB? Most linux distros command line utils/editors support auto decompress of gzipped files. But I guess that doesn't apply to windows very well. Since all my EDA work is on linux (none of the asic tools support windows), I tend to foget it exists for other eda industries like PCB.

     

    I don't think you can claim checking them into a revision control system as a feature/positive. The text in a file can look completely different to "diff", but still represent the exact same design data. The same net coordinates can live on line 5 or line 5,000,000 and still represent the same thing. We check in our text verilog netlists to RCS at work, but the diffs usually end up being 95% different despite only having 3 new logic gates added. Might as well been a binary file at those diff rates. Now if you can guarentee exact text formatting and order from output to output, then this diff issue obviously goes away.

     

    That being said, it is great when I run across an EDA vendor using a straight forward text file for a input or output file. Did you board dimentions change, but you want to keep all the x,y coordinates of your components? Start up a new board and draw the dimenstions, then save to a text file. Rip out the old componet coordinate section from the old text file and and place it in the new one and vola! Instant component placements! ( this is a somewhat bad example since you can just take your board file and edit the dimensions with components in place - but I think you get my point).

     

    BTW - I love eagle cad. Any chance you guys could get a graphics designer to re-work your icons? Nothing says polished tool like professional and modern icons.

  • I'm cautiously optimistic that they'll do the right thing.  The major (ab)uses of XML can get away with the mutated forms they do precisely because they are major.  EDA doesn't have that luxury.

     

    I've found that XML works best for separating form (e.g. SPICE vs CDL vs Verilog language quirks) from function (i.e. instantiation, connectivity, parametrization, etc). Its strengths really shine when you need to do complex scripted interaction with the data.  Pick a language (aside from Cadence's SKILL ) and you'll find at least basic XML parsing hooks.  No need for custom regexp, antlr, javacc, binary-reading parsers for the N+1th platform variation.  You can even test XSLT works-in-progress via your browser of choice!

  • XML diff works best if you enforce lang-based sorting of the hierarchy.  For example, if you have a lib-cells-views-insts structure, the cells will always appear in the same cell-name alphanum order, the views will always appear in the same view-name alphanum order, and most importantly all the instances will appear in the inst-name alphanum sorted order.  For coordinates-based diffs, graphical structures could stick to a numerically-sorted order, like X-coord then Y-coord, or worst case a brute-force XOR/intersection operation using something like Java's built-in Area class boolean ops.

     

    Worst-case, build the enforced sorting mechanism into the revision control front-end.  There's always a way!

  • I think having a text based format would be a real win for Cadsoft, it would bring greatly desired version control into our process scope.Being able to edit the files also adds to the version control benefits for community (OS) based work. As for the format I can see why some are afraid of XML as it can be bulky compared to simpler text formats. However XML does allow for powerful features like schema to reduce ambiguity and provide readable open standards. It also allows for easy transformation/conversion to other XML standards using standard XSL tools. Most programming languages also have good XML libraries opening the eagle files to an even wider audience for hacking and manipulation.

     

    As pointed out above by yzf600 size can be a real issue with larger netlists and projects, xml file editing with larger files is very difficult for a human. not so much on the parsing side for SAX based libraries. Probably the best way to alleviate size issue is to support and promote modularity. Modularity in Eagle allows sections of a design to be copied and pasted into new or existing projects, it also allows for repetitive reproduction of identical sections within a circuit/design. This can be represented in XML using XLinks pointing to accompanying XML modules. These modules get linked in during parsing. This is much more efficient as it reduces duplication of XML and allows for tree structures of a design. Such a standard could also allow for hardware licenses and or copyright/copyleft inclusion to make the entire sharing process much simpler. Modularity of this type would be a boon for community and opesource development within Eagle whilst simultaneously keeping file size under control. It also makes version control even easier particularly if the Xlink usage includes remote Git or subversion hrefs (via the standard url identity) allowing for remote version controlled modularity, something that the hardware opensource community would love to have.

     

    regards

    Al

  • is there some news about version 6.0 from cadsoft?

  • I moved over to gEDA back in '03 and while it is cool to see vendors now distribute part footprints and symbols in Eagle format I really want a compatable option. While I am really happy to see CadSoft sticking there neck out on this  thing I have to wonder why I took so long? Why didn't it happen the same way gerber format did? I can see how no cad vendor wanted it because vendor lock in has a lot of appeal to them. Next we need a wikipedia like thing with all the various footprint files with written discriptions and compatability charts.

     

    I have been frequenctly irritated by the amount of open source  hardware that has it's schematics distributed in Eagle's binary format. I know the user interface for gEDA is an aquired tast and I am not trying to push it on everyone but I think the symbol file format which it text based is worth a group look. The footprint file format that PCB uses is nice but it lacks a lot of things I would really like. I hope they don't just try to build one giant format. I would rather see seperate XML formats for part footprint, mechanical 3d info, symbol, schematic and so on.

  • What license are they going to put on the final specification? Since this is all about the open hardware someone had to ask.

1 2 3 Previous Next

Related Content


Related Products
Discussions
  • Retrieving data ...

Bookmarked By (0)