Re: [Bug-apl] ISO Component File API preview

[David Lamkins]

No reason to not speak plainly. We're all working toward similar goals. Only the details differ. But the devil is in the details, isn't it...? :)

For brevity, I've taken the liberty of excerpting your responses.

On Thu, Jul 10, 2014 at 2:22 PM, Blake McBride <address@hidden> wrote:

Okay, I read the spec.  It is way too minimalistic IMO.  It would be best to make our component file system a super-set of that definition.

On the other hand, it's a spec that ought to have some kind of implementation. That is my goal. I've been careful to implement that spec as faithfully as I'm able.
 

Their "Library" should map to an SQL database.

Their "file" should map to an SQL table.

This is our main point of contention. I agree, absent having done the work and discovering the edge cases, that the spec's terminology *could* map to corresponding SQL entities. I remain unconvinced that this *should* be the case.
 

Satisfying the spec is insufficient.  Your satisfaction of the spec must also be reasonable.

That's a matter of opinion. I happen to believe that satisfying the spec is an excellent starting position.

 

 It is unreasonable to store one table per SQL database.

While that is certainly true in the general sense of SQL database, that's not what I'm actually doing even given my use of SQLite. As I stated several messages previous to this, SQLite is *not* a database server. SQLite is a library which directly manipulates its database file. There is no server. To use an analogy: SQLite is no more a database server than the C stdio routines are a file server.
 

But *file* is an abstract notion.

There certainly are layers of abstraction involved in the implementation of a filesystem. But the operations one performs on a file are distinct from the operations one performs on various database entities.

Believe me, I appreciate that one may invent a mapping from "file" to "database". I'm just not convinced that it needs to be done in this case.

 

 Creating a native component file system is likely more effecient than riding on top of a huge database engine.  That is why all of the vendors you quote did that.  

You're overlooking the fact that a file-based system is much easier for the average application programmer to use, given that there's no need to install, configure and administer a database server.

You're also ignoring the fact that the major APL vendors offer both a component file API and a database API.

 

The problem is that developing a native component file system is significantly more work than riding on top of SQL.

That's true. It's also worth the effort for the reasons you listed earlier.
 

A component file system written in APL that rides on top of an SQL engine is reasonable.

I certainly think so. It saved me a lot of difficult work. That said, I wouldn't have made that choice if not for manner in which SQLite is implemented.
 

 A native component file system implementation would have to be written in something like C.  That would be somewhat unreasonable in APL.

I don't equate difficult and time-consuming with unreasonable. Yes, it'd be more work. Yes, it'd be more difficult to get "right". However, I believe that this is the correct long-term approach for a performant component file system.

I'm offering the current code because it can serve as a reliable stopgap until such time as something better comes along. I can envision (but don't promise to write) a native component file system that doesn't have the performance issues caused by the implicit transactions inherent in SQLite.

 

I've used many component file system over the years, and I've written my own file systems, so I am familiar with what they are in general terms.  Having now read the spec, it is even less than I would have expected.

That's unsurprising. I, too, would've liked to see more elaboration w.r.t. things like sharing and access control. But it is what it is... The question on table - at least as far as I'm concerned - is whether or not the current implementation satisfies the existing spec.

 

Are you mapping one APL component file into one SQL database?  (This is what I understanding of how your system is working.  If you answer this one question, I can reply better.)

Unfortunately, you're ignoring the way in which SQLite functions; you won't be happy with my answer until you take some time to understand the difference between SQLite and a "real" database engine.

Here's a good starting point: <http://www.sqlite.org/about.html>. The first two paragraphs are the most important.

That said, I am not "mapping one APL component file into one SQL database" within the context that you're applying.

 

I hope you don't take my directness as disrespectful.  I've seen some of your work, and I respect and appreciate it.

Thanks. Likewise w.r.t. your contributions to this community.

We're just a couple guys trying to hash out some concepts from very different perspectives.

 

--

"The secret to creativity is knowing how to hide your sources."

   Albert Einstein

http://soundcloud.com/davidlamkins
http://reverbnation.com/lamkins
http://reverbnation.com/lcw
http://lamkins-guitar.com/
http://lamkins.net/
http://successful-lisp.com/