I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.


The problem: "How do I make an existing linux file server be a private
cloud file server, targeting other linux, ChromeBooks and Android clients?"


There is a LOT of stuff that misses out of critical features - or those
features need a paid-for version that costs $1700-9000 per year.

I've played with Tonido and OwnCloud so far.

Obvious omissions in the free versions:

a Limited shares (OwnCloud only seems to support 1 exported directory -
I have lots);

b (Serious) Do not integrate with POSIX users. The idea is the file
sharing should integrate with linux, not try and take over.


I did look at OpenAFS but there's no Android client.


SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.


OK - allowing for the fact that "I could write it myself dammit"
(actually, no, I don't have that type of coding skill) I'm very
surprised such a killer app is missing from the opensource stable.

This is the same opensource that brought us *BSD, Linux, perl, python, 2
super RDBMSs, top rate image editing, rock solid *connected*
fileservices (NFS, SMB) allowing for the fact the latter had to be
reverse engineered.

Have all the creative types died?

On 2015-03-04 14:14, Tim Watts wrote:
I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.


The problem: "How do I make an existing linux file server be a private
cloud file server, targeting other linux, ChromeBooks and Android clients?"


There is a LOT of stuff that misses out of critical features - or those
features need a paid-for version that costs $1700-9000 per year.

I've played with Tonido and OwnCloud so far.

Obvious omissions in the free versions:

a Limited shares (OwnCloud only seems to support 1 exported directory -
I have lots);

b (Serious) Do not integrate with POSIX users. The idea is the file
sharing should integrate with linux, not try and take over.


I did look at OpenAFS but there's no Android client.


SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.


OK - allowing for the fact that "I could write it myself dammit"
(actually, no, I don't have that type of coding skill) I'm very
surprised such a killer app is missing from the opensource stable.

This is the same opensource that brought us *BSD, Linux, perl, python, 2
super RDBMSs, top rate image editing, rock solid *connected*
fileservices (NFS, SMB) allowing for the fact the latter had to be
reverse engineered.

Have all the creative types died?


Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

Jeroen Belleman

On 04/03/15 14:15, Jeroen Belleman wrote:
On 2015-03-04 14:14, Tim Watts wrote:
I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.


The problem: "How do I make an existing linux file server be a private
cloud file server, targeting other linux, ChromeBooks and Android
clients?"


There is a LOT of stuff that misses out of critical features - or those
features need a paid-for version that costs $1700-9000 per year.

I've played with Tonido and OwnCloud so far.

Obvious omissions in the free versions:

a Limited shares (OwnCloud only seems to support 1 exported directory -
I have lots);

b (Serious) Do not integrate with POSIX users. The idea is the file
sharing should integrate with linux, not try and take over.


I did look at OpenAFS but there's no Android client.


SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.


OK - allowing for the fact that "I could write it myself dammit"
(actually, no, I don't have that type of coding skill) I'm very
surprised such a killer app is missing from the opensource stable.

This is the same opensource that brought us *BSD, Linux, perl, python, 2
super RDBMSs, top rate image editing, rock solid *connected*
fileservices (NFS, SMB) allowing for the fact the latter had to be
reverse engineered.

Have all the creative types died?


Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

Jeroen Belleman

Indeed. I find that a 'remote backup of local data' which allows global
access to that data is easily achieved with rsync.

Assuming Linux CLIENTS


For 'within these walls' services I simply have a file server running
NFS and samba which covers all the bases.

And its own internal backup policy and a nice minidnla server to export
videos.


--
Everything you read in newspapers is absolutely true, except for the
rare story of which you happen to have first-hand knowledge. €“ Erwin Knoll

On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

Because that's what I want.

And I have not found any.

SMB - no chance
NFS - not the slightest bit suitable
OpenAFS - the right logic is there but no client support

Most of the cloud-fs solutions seem to hang around WebDAV - but probably
augmented with update hints to aid caching and push delivery of updates.


It is a specialist network filesystem - that is for sure. It strikes me
as strange that there are 101 commercial suppliers (Dropbox, Google,
Evernote, Spideroak...) and yet no one has come up with a nice well
defined open protocol that addresses replication and syncing.

On 04/03/15 14:19, The Natural Philosopher wrote:

Indeed. I find that a 'remote backup of local data' which allows global
access to that data is easily achieved with rsync.

Assuming Linux CLIENTS


For 'within these walls' services I simply have a file server running
NFS and samba which covers all the bases.

And its own internal backup policy and a nice minidnla server to export
videos.


And that does not meet my criteria.

It's kind of what I do with a laptop - but with Unison rather than rsync.


But when you have TBs of data (many media files - mostly photos) you
will never be able to sync that 100% to random devices - they simply
don't have the storage.

At Wed, 04 Mar 2015 14:32:04 +0000 Tim Watts wrote:


On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

Because that's what I want.

And I have not found any.

SMB - no chance
NFS - not the slightest bit suitable
OpenAFS - the right logic is there but no client support

Most of the cloud-fs solutions seem to hang around WebDAV - but probably
augmented with update hints to aid caching and push delivery of updates.

What about something like git or subversion? *Subversion* uses a WebDAV-based
protocol for svn-over-http(s).



It is a specialist network filesystem - that is for sure. It strikes me
as strange that there are 101 commercial suppliers (Dropbox, Google,
Evernote, Spideroak...) and yet no one has come up with a nice well
defined open protocol that addresses replication and syncing.

What is sounds like is that you are looking for is something like subversion's
WebDAV-based protocol. The bits and pieces are there. WebDAV is reasonably
well documented. Subversion is open source, so one can have a look at what it
does.

As others have said, programs like rsync seem to cover the bases that Linux
*Developers* are interested in. I suspect that few of the Linux *developers*
are interested in yet another pointy-clicky 'toy'. *I* certainly have no
interest in such a toy. OTOH, I have dial-up Internet access at home, so any
sort of 'Cloud' services are pretty much not much use to me. Subversion (or
git) are about the most I can deal with.





--
Robert Heller -- 978-544-6933
Deepwoods Software -- Custom Software Services
http://www.deepsoft.com/ -- Linux Administration Services
-- Webhosting Services

Tim Watts writes:

On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

Because that's what I want.

And I have not found any.

SMB - no chance

Both my Kindle and Android have no problem with Samba shares over
wireless.

--
Dan Espen

On 04/03/15 14:47, Jethro_uk wrote:

Not quite sure what your need is, but for some reason I've remembered
that it's possible to run a single RAID across many devices under
mdadm ... never done it, but would that help ?


No. The clients include Android with tiny local storage.


I'll restate the requirements:

1) Central server;

2) Internet access;

3) Android and Chromebook clients with local caching (like Google Drive)

4) Linux client (obviously)


=======

It's not actually outlandish. If it were, there would not be a load of
companies doing it. In fact it is a very well defined problem.

However, some of us prefer to keep out data at home (not to mention that
TBs of data in the public cloud is actually not cheap!).

Remember that whilst lots of us do have TBs of data, we only access a
small amount at any time.

eg: I might mark my "documents" folder and "data/house-plans" folder to
be fully sync'd (offline mode) as I access spreadsheets and "word"
documents in both very regularly. I might mark one photo sub directory
as "sync offline mode" if I were going to show some stuff at a remote site.

The rest I am happy to take my chances of internet connectivity.

It could also be viewed as an HSM problem...

In fact I'm surprised that Western Digital or Buffalo have not sponsored
some development because it's exactly what you'd want for a little
NAS/cloud box.

I also wonder why noone's tried reverse engineering GoogleDrive protocol
- a lot of effort was made to hack SMB. That way, the clients are a done
deal and Google Drive works really well[1] - I can edit a document on my
computer and watch it update automagically on my phone.

[1] apart from the fact that free storage is untrustworthy, both for
reliability and security and buying TBs would cost.

On 04/03/15 14:53, Robert Heller wrote:
At Wed, 04 Mar 2015 14:32:04 +0000 Tim Watts wrote:


It is a specialist network filesystem - that is for sure. It strikes me
as strange that there are 101 commercial suppliers (Dropbox, Google,
Evernote, Spideroak...) and yet no one has come up with a nice well
defined open protocol that addresses replication and syncing.

What is sounds like is that you are looking for is something like subversion's
WebDAV-based protocol. The bits and pieces are there. WebDAV is reasonably
well documented. Subversion is open source, so one can have a look at what it
does.

Hmm - I do wonder how that would scale with big binary files. But I
agree, there is almost certainly something useful in the meta data that
would assist sync-ing. if we dispensed with the ability to roll back and
simply worried about every client's idea of the HEAD version, then it
would be easy for clients to work out who had the most recent copy.

But I think the mercurial logic would be better suited compared to SVN
as Mercurial is an HTTP based version of Git and Git was designed from
the ground up to be devoid of any central authority.

As others have said, programs like rsync seem to cover the bases that Linux
*Developers* are interested in. I suspect that few of the Linux *developers*
are interested in yet another pointy-clicky 'toy'. *I* certainly have no
interest in such a toy.

With respect I would hardly describe it as a "toy"

It is a real world problem.

Case in point:

10 years ago my house had desktops, connected by NFS to a central server
and all was well.

Now, we have people (4) passing laptops, pads and phones around. And we
may be needing a chromebook for the kids (schooling). NFS simply does
not work.

I find myself uploading stuff ad hoc to Google Drive just so SWMBO can
look at a copy on her phone.

if I don't get a solution in place we will have 7 devices with random
bits of random peoples files all over them.

Open source dev folk use pads and phones - maybe not Chromebooks - but I
refuse to believe they are so stuck in the stone age they cannot see
where the future is going. Right now it's a mess.

This is most definitely a problem that needs a solution even if not
everyone knows it yet.

On 04/03/2015 13:14, Tim Watts wrote:
I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.


The problem: "How do I make an existing linux file server be a private
cloud file server, targeting other linux, ChromeBooks and Android clients?"


There is a LOT of stuff that misses out of critical features - or those
features need a paid-for version that costs $1700-9000 per year.

I've played with Tonido and OwnCloud so far.

Obvious omissions in the free versions:

a Limited shares (OwnCloud only seems to support 1 exported directory -
I have lots);

b (Serious) Do not integrate with POSIX users. The idea is the file
sharing should integrate with linux, not try and take over.


I did look at OpenAFS but there's no Android client.


SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.


OK - allowing for the fact that "I could write it myself dammit"
(actually, no, I don't have that type of coding skill) I'm very
surprised such a killer app is missing from the opensource stable.

This is the same opensource that brought us *BSD, Linux, perl, python, 2
super RDBMSs, top rate image editing, rock solid *connected*
fileservices (NFS, SMB) allowing for the fact the latter had to be
reverse engineered.

Have all the creative types died?


I use Open Media Vault on a HP Microserver.

WebDAV might give you what you want

Chromebooks aren't very friendly to anything that isn't Google Drive,
although I have used webdav with mine.

On 04/03/15 14:57, Dan Espen wrote:
Tim Watts writes:

SMB - no chance

Both my Kindle and Android have no problem with Samba shares over
wireless.


OK - I'm interesed - please tell me more.

What client are you using and can it offline sync certain files?

That was an assumption on my part - I have not seen SMB jump out at me
with my interaction with Android.

However, it's no bother to install SaMBa on teh servers if it solves the
problem... At least I know it works correctly at the back end (user
auth, POSIX UID/GID mapping etc).

Cheers!


Tim

On 04/03/2015 14:57, Dan Espen wrote:
Tim Watts writes:

On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

Because that's what I want.

And I have not found any.

SMB - no chance

Both my Kindle and Android have no problem with Samba shares over
wireless.


My Xperia Android devices even have an option to mount SMB shares as if
they were SD cards. Bit surprising to find that feature.

On 04/03/15 15:10, HarpingOn wrote:

I use Open Media Vault on a HP Microserver.

That's interesting - I have not come across that. I will spin that up on
a virtual machine and see who it works - if it does, I can probably
cherry pick the bit I want and shove it onto my existing Debian server
(which coincidently runs on an HP Micro Server!).

WebDAV might give you what you want

May I ask - what client are you using? I tried that once (about a year
back) but the couple of clients I tried were ropey.

Chromebooks aren't very friendly to anything that isn't Google Drive,
although I have used webdav with mine.

That is also very useful - thank you.

On 04/03/15 15:11, HarpingOn wrote:
On 04/03/2015 14:57, Dan Espen wrote:

My Xperia Android devices even have an option to mount SMB shares as if
they were SD cards. Bit surprising to find that feature.

How do they respond when the network goes wobbly? Graceful failure and
back up when the network is stable again?

On 04/03/15 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

Because that's what I want.

And I have not found any.

SMB - no chance
NFS - not the slightest bit suitable

https://play.google.com/store/apps/d...smanager&hl=en


OpenAFS - the right logic is there but no client support

Most of the cloud-fs solutions seem to hang around WebDAV - but probably
augmented with update hints to aid caching and push delivery of updates.


It is a specialist network filesystem - that is for sure. It strikes me
as strange that there are 101 commercial suppliers (Dropbox, Google,
Evernote, Spideroak...) and yet no one has come up with a nice well
defined open protocol that addresses replication and syncing.


This is so much a wintroll type post.

"I want EXACTLY what I used to have in Windows for nothing on Linux, and
if Linux can't do it exactly .linux sucks'

In short if you want a commercial cloud behaviour stick to commercial
clouds.




--
Everything you read in newspapers is absolutely true, except for the
rare story of which you happen to have first-hand knowledge. €“ Erwin Knoll

In uk.comp.os.linux Tim Watts wrote:
On 04/03/15 15:10, HarpingOn wrote:

I use Open Media Vault on a HP Microserver.

That's interesting - I have not come across that. I will spin that up on
a virtual machine and see who it works - if it does, I can probably
cherry pick the bit I want and shove it onto my existing Debian server
(which coincidently runs on an HP Micro Server!).

What's interesting about OMV in regards to this discussion? It looks like
A.N.Other NAS Linux distro. I can't see anything there that handles offline
files, which is the critical part of the problem.

There is an OpenAFS FUSE client, so that could work for Android, Mac, etc
though it's still experimental (no write support). And no pretty GUI.

Very interesting thread, BTW - I have exactly the same problems...

Theo

On 04/03/15 15:36, The Natural Philosopher wrote:

https://play.google.com/store/apps/d...smanager&hl=en

Thank you - will check that out later.

This is so much a wintroll type post.

"I want EXACTLY what I used to have in Windows for nothing on Linux, and
if Linux can't do it exactly .linux sucks'

Now you're making me laugh - because you of all people know that I am a
linux die hard!

Note I am not criticising linux, but making a pertinent observation
that, after all the massive effort that went into cracking SMB (and I
remember that effort, late 90's) I am SO surprised that noone (not even
an MIT student project) has come along and said - this is actually not
so hard, and just did it.

Bit like the way we all put up with that PoS WU-IMAPd then Timo Sirainen
came along and said "this is ******** - here I just wrote dovecot and
solved all the logic problems cleanly".

Ditto Philip Hazel and exim.


The complication here is the massive array of client devices that need
supporting. It would probably be a good approach to be able to emulate a
Google Drive server protocol (then we don't care about teh client
software because it exists).

There is a strong risk that Google will then say "**** you, we just
upgraded the protocol". However, by that time, if the server is proven,
people would have a chance to write replacement clients without it being
an "all at once" problem.



In short if you want a commercial cloud behaviour stick to commercial
clouds.

That was rather the point. Those guys have proved it can be done well.
But there needs to be an opensource version that works as well (and
there isn't).

On 04/03/15 15:56, Tim Watts wrote:
On 04/03/15 15:36, The Natural Philosopher wrote:

https://play.google.com/store/apps/d...smanager&hl=en


Thank you - will check that out later.

This is so much a wintroll type post.

"I want EXACTLY what I used to have in Windows for nothing on Linux, and
if Linux can't do it exactly .linux sucks'

Now you're making me laugh - because you of all people know that I am a
linux die hard!

Note I am not criticising linux, but making a pertinent observation
that, after all the massive effort that went into cracking SMB (and I
remember that effort, late 90's) I am SO surprised that noone (not even
an MIT student project) has come along and said - this is actually not
so hard, and just did it.

Bit like the way we all put up with that PoS WU-IMAPd then Timo Sirainen
came along and said "this is ******** - here I just wrote dovecot and
solved all the logic problems cleanly".

Ditto Philip Hazel and exim.


The complication here is the massive array of client devices that need
supporting. It would probably be a good approach to be able to emulate a
Google Drive server protocol (then we don't care about teh client
software because it exists).

There is a strong risk that Google will then say "**** you, we just
upgraded the protocol". However, by that time, if the server is proven,
people would have a chance to write replacement clients without it being
an "all at once" problem.



In short if you want a commercial cloud behaviour stick to commercial
clouds.

That was rather the point. Those guys have proved it can be done well.
But there needs to be an opensource version that works as well (and
there isn't).

There are some very good reasons for that.

An internet based cloud is by definition reliant on a high bandwidth
server somewhere on the internet backbone.

At some level that is a commercial entity.

Adding free software to a rented (virtual) server to make a global cloud
is not actually any cheaper than renting a bit of someone elses cloud.

If its domestic file sharing you want, stuff a domestic files server in.



--
Everything you read in newspapers is absolutely true, except for the
rare story of which you happen to have first-hand knowledge. €“ Erwin Knoll

On 04/03/15 15:54, Theo Markettos wrote:
In uk.comp.os.linux Tim Watts wrote:
On 04/03/15 15:10, HarpingOn wrote:

I use Open Media Vault on a HP Microserver.

That's interesting - I have not come across that. I will spin that up on
a virtual machine and see who it works - if it does, I can probably
cherry pick the bit I want and shove it onto my existing Debian server
(which coincidently runs on an HP Micro Server!).

What's interesting about OMV in regards to this discussion? It looks like
A.N.Other NAS Linux distro. I can't see anything there that handles offline
files, which is the critical part of the problem.

I was assuming they had a tuned WebDav server instance - that might be
worth ripping off as it's probably only going to be an apache config.

I was hoping that there might be a client that could handle the caching
- though without certain explicit server support it may not be very
efficient.


There is an OpenAFS FUSE client, so that could work for Android, Mac, etc
though it's still experimental (no write support). And no pretty GUI.
Hmm...


Very interesting thread, BTW - I have exactly the same problems...

Thank you - I am surprised it's only the two of us though. Perhaps
others are less picky?

The commercial guys clearly see a LOT of milage in this. However, even
Tonido came back to me and said their Enterprise FileCloud (that costs
$$$$) can only handle NTFS user file perms and ownership - POSIX may
happen in the future.


I have seen this moment before, at least twice:

1) The great MS office desert - if you were on linux/BSD and everyone
else was throwing Word files around. It was a long time before something
better than "strings" came along and it was largely thanks to Sun.

2) Email. There was a very dark time in the late 90's/early 00's when we
had a **** poor IMAP server (WU-IMAPd) and very ropey client support.

Then Thunderbird got its act together and Timo knocked out dovecot and
all because sane again.

SMTP was less of a problem even before Philip Hazel as sendmail works,
even though it had a config from hell.


It does seem like many of the great solutions are down to one person,
Linus, Timo, Philip. It can take a while for the momentum to roll.

In some cases, a great project simply sits at risk of dying. case in
point: MoSH - https://mosh.mit.edu/ - I use it all the time - it is
fantastic when you have a tethered internet connection with the IP
changing all the time. However, it has not gained much traction and even
though adding new features like tunnels has been mooted, noone wants to
do it and I expect the original guys are off doing other interesting stuff.

"The Natural Philosopher" wrote in message
...
On 04/03/15 14:15, Jeroen Belleman wrote:
On 2015-03-04 14:14, Tim Watts wrote:
I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.


The problem: "How do I make an existing linux file server be a private
cloud file server, targeting other linux, ChromeBooks and Android
clients?"


There is a LOT of stuff that misses out of critical features - or those
features need a paid-for version that costs $1700-9000 per year.

I've played with Tonido and OwnCloud so far.

Obvious omissions in the free versions:

a Limited shares (OwnCloud only seems to support 1 exported directory -
I have lots);

b (Serious) Do not integrate with POSIX users. The idea is the file
sharing should integrate with linux, not try and take over.


I did look at OpenAFS but there's no Android client.


SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.


OK - allowing for the fact that "I could write it myself dammit"
(actually, no, I don't have that type of coding skill) I'm very
surprised such a killer app is missing from the opensource stable.

This is the same opensource that brought us *BSD, Linux, perl, python, 2
super RDBMSs, top rate image editing, rock solid *connected*
fileservices (NFS, SMB) allowing for the fact the latter had to be
reverse engineered.

Have all the creative types died?


Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

Jeroen Belleman

Indeed. I find that a 'remote backup of local data' which allows global
access to that data is easily achieved with rsync.

Assuming Linux CLIENTS


For 'within these walls' services I simply have a file server running NFS
and samba which covers all the bases.

And its own internal backup policy and a nice minidnla server to export
videos.


--
Everything you read in newspapers is absolutely true, except for the rare
story of which you happen to have first-hand knowledge. - Erwin Knoll

Yep - same here, works good - two external drives and it's internal drive -
triple backup, totally effortless

On 04/03/15 16:05, The Natural Philosopher wrote:

If its domestic file sharing you want, stuff a domestic files server in.

Exactly.

and now what software shall I use?

I have enough bandwidth on ADSL. I even have VDSL available as soon as I
am ready to migrate (that'll give me about 60/15Mbit/s)

Bandwidth is not an issue - protocol and software is.

On 04/03/15 16:11, John F wrote:

Yep - same here, works good - two external drives and it's internal drive -
triple backup, totally effortless



And not solving my problem (I too have a nice "internal" solution - and
now that needs to evolve...)

Tim Watts writes:

On 04/03/15 14:57, Dan Espen wrote:
Tim Watts writes:

SMB - no chance

Both my Kindle and Android have no problem with Samba shares over
wireless.


OK - I'm interesed - please tell me more.

What client are you using and can it offline sync certain files?

ES3.

Maybe I'm not clear on what you want.
ES3 allows the devices to mount Samba shares.

I have a large directory full of music.
Either device can mount the share and play the music.

If I wanted to sync to the device, I'd just run rsync on either end.

I use something different for photo sharing over USB.
Since the photos may appear as a filesystem or using MTP,
I use Gphoto2 to copy from device to Linux hard disk or
to remove photos I don't want from the device.

That was an assumption on my part - I have not seen SMB jump out at me
with my interaction with Android.

Android being Linux can run a Samba client just fine.
There's more than one Samba client as I remember.

However, it's no bother to install SaMBa on teh servers if it solves
the problem... At least I know it works correctly at the back end
(user auth, POSIX UID/GID mapping etc).

You can run Samba and/or NFS. Windows can also do NFS
(I read somewhere), but Samba is more universal.

--
Dan Espen

The Natural Philosopher writes:

This is so much a wintroll type post.

Maybe, but I prefer to see it as a technical question.
No need to be antagonistic, even if the post reads like
an attempt to push some peoples buttons.

I think there is a simple solution to the requirements as
I understand them.

--
Dan Espen

On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with for
~£60pa and you get office free on 5 PCs for that.

On 04/03/15 16:57, Dan Espen wrote:
Tim Watts writes:

On 04/03/15 14:57, Dan Espen wrote:
Tim Watts writes:

SMB - no chance

Both my Kindle and Android have no problem with Samba shares over
wireless.


OK - I'm interesed - please tell me more.

What client are you using and can it offline sync certain files?

ES3.

Maybe I'm not clear on what you want.
ES3 allows the devices to mount Samba shares.

Thanks - that might be worth checking out as a general file manager (I
am using "File Manager" right now).

I have managed SFTP access with mine - but the main sticking point is
you cannot mark files for offline access (aka Keep on Device").

The whole "mount SFTP/SMB" thing is an acceptable last resort - but I
really would like to have user selectable caching as that makes a big
difference in usability if the network goes away.

On 04/03/15 17:07, Dennis@home wrote:
On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with for
~£60pa and you get office free on 5 PCs for that.

And unfortunately it's not under my control. Do I want pictures of my
kids swimming on their server?

On 04/03/15 17:01, Dan Espen wrote:
The Natural Philosopher writes:

This is so much a wintroll type post.

Maybe, but I prefer to see it as a technical question.
No need to be antagonistic, even if the post reads like
an attempt to push some peoples buttons.

Well - OK, I pushed some buttons. But hey, it got a lively debate with
some interesting ideas... But no, I am hard core Linux, but not so hard
core that I fail to see the world changing


I think there is a simple solution to the requirements as
I understand them.


It would look that way - which is why I am amazed that in weeks (ad hoc)
of googling I have not found a really good solution yet.

I've found lots of 50-80% solutions though...

Technically Google Drive and Dropbox have 100% but fail on the "not
private, and not trustworthy" criteria. But they prove the prove the
problem is solvable.

The commercial software guys (Tonido, Owncloud etc) just don't quite get
it - they seem to be suffering from the "those who don't understand unix
are doomed to reinvent it - poorly" syndrome.

The key point is "multiple users, POSIX/NTFS+AD mapping" - Oooh wouldn't
that be a good idea... Doh.

I do understand that the community/free versions will be clipped but to
have nothing between that and a $9000 50 user/year licence fee is insane
(yes, one of them charges that much). Even VMWare understood the concept
of a hobbyist's license and those guys are not known for being cheap.

On 04/03/15 17:18, Huge wrote:
On 2015-03-04, Tim Watts wrote:

[stuff]

Shouldn't you have cross-posted this?



yeah - It's TNP's client keeps trimming the groups and I usually notice,
but not this time...

I did read up on OpenAFS BTW - but the lack of Android client is a bit
of a problem. Pity 'cos I already use Kerberos and that's one major
hurdle gone...

On 04/03/15 17:23, Tim Watts wrote:
On 04/03/15 17:18, Huge wrote:
On 2015-03-04, Tim Watts wrote:

[stuff]

Shouldn't you have cross-posted this?



yeah - It's TNP's client keeps trimming the groups and I usually notice,
but not this time...


Not my client, my news server (albasani) that doesnt like cross posts
without a follow-up set


--
Everything you read in newspapers is absolutely true, except for the
rare story of which you happen to have first-hand knowledge. €“ Erwin Knoll

On 04/03/15 18:00, The Natural Philosopher wrote:
On 04/03/15 17:23, Tim Watts wrote:
On 04/03/15 17:18, Huge wrote:
On 2015-03-04, Tim Watts wrote:

[stuff]

Shouldn't you have cross-posted this?



yeah - It's TNP's client keeps trimming the groups and I usually notice,
but not this time...


Not my client, my news server (albasani) that doesnt like cross posts
without a follow-up set



OK - please accept my apologies. I'll just keep adding them back..

On 04/03/2015 17:15, Tim Watts wrote:
On 04/03/15 17:07, Dennis@home wrote:
On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with for
~£60pa and you get office free on 5 PCs for that.

And unfortunately it's not under my control. Do I want pictures of my
kids swimming on their server?


Encryption is your friend.

On 04/03/15 18:43, Dennis@home wrote:
On 04/03/2015 17:15, Tim Watts wrote:
On 04/03/15 17:07, Dennis@home wrote:
On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with for
~£60pa and you get office free on 5 PCs for that.

And unfortunately it's not under my control. Do I want pictures of my
kids swimming on their server?


Encryption is your friend.

Not really.

You are missing the point I fear - this has to be seamless and usable by
anyone.

Unless you know of an file manager app that does on the fly encryption?

On 04/03/15 13:14, Tim Watts wrote:
I have raised this before... And tried some stuff. And I cannot believe
there is no solution... hence the first part of the subject.

OK - much interesting discussion.

But I am trying to throw this to Ask Slashdot as I think it has merit.

anyone who feels so interested or inclined, please upvote my submission:

http://rpx.me/3DL7l

Ta

Tim

On 04/03/2015 18:55, Tim Watts wrote:
On 04/03/15 18:43, Dennis@home wrote:
On 04/03/2015 17:15, Tim Watts wrote:
On 04/03/15 17:07, Dennis@home wrote:
On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with
for
~£60pa and you get office free on 5 PCs for that.

And unfortunately it's not under my control. Do I want pictures of my
kids swimming on their server?


Encryption is your friend.

Not really.

You are missing the point I fear - this has to be seamless and usable by
anyone.

Unless you know of an file manager app that does on the fly encryption?

https://www.boxcryptor.com/en/onedrive

On 04/03/15 19:13, Dennis@home wrote:
On 04/03/2015 18:55, Tim Watts wrote:

Unless you know of an file manager app that does on the fly encryption?

https://www.boxcryptor.com/en/onedrive

Wow. Was not expecting that.

That is actually a potentially serious nutcracker there.

Thank you.

If not my entire media collection, that could be a good contender for
shared documents (finances, etc).

Tim Watts writes:

On 04/03/15 17:07, Dennis@home wrote:
On 04/03/2015 14:32, Tim Watts wrote:
On 04/03/15 14:15, Jeroen Belleman wrote:

Bah. Linux and Unix have all sorts of remote/shared filesystems.
Of course, some 'creative' minds thought it'd be a good idea to
invent yet another such thing. That's the bane of computing. It's
so often easier to reinvent something than it is to find out how
to use the existing tricks.

OK - tell me one that works well with Android, Chromebooks with a
semi-connected internet?

M$ one drive works quite well with android, you get 5TB to play with for
~£60pa and you get office free on 5 PCs for that.

And unfortunately it's not under my control. Do I want pictures of my
kids swimming on their server?

Huh?

What exactly are you trying to protect? Are your kids especially ugly?

I've taken plenty of pictures of people in the water.
I don't see the issue.


--
Dan Espen

On 04/03/15 19:51, Dan Espen wrote:


Huh?

What exactly are you trying to protect? Are your kids especially ugly?

I've taken plenty of pictures of people in the water.
I don't see the issue.



It's not a big surprise, someone not trusting external cloud providers.
Have you not seem the amount of pics hacked off various photo sharing
sites that the user really did not want to have in the public domain?

So it's not something I need to defend really - I just don't want all my
personal stuff on someone else's servers. I have no problem with using
Google Calendar because I never have anything that interesting in there.
But if I were running an innovative company and used it to store all
sorts if meetings with interesting people, then I would think twice.

In uk.d-i-y Tim Watts wrote:
On 04/03/15 19:13, Dennis@home wrote:
On 04/03/2015 18:55, Tim Watts wrote:

Unless you know of an file manager app that does on the fly encryption?

https://www.boxcryptor.com/en/onedrive

Wow. Was not expecting that.

That is actually a potentially serious nutcracker there.

Thank you.

If not my entire media collection, that could be a good contender for
shared documents (finances, etc).

As I see it there are two parts to the problem: storage and syncing.

Storage is covered by the NAS. That's the master copy, everything should
migrate there. You can nuke all the other copies in the world and you'll
only lose the most recent changes. You can do a bulk download
quite easily over gigabit ethernet/wifi/USB/SATA.

Tools like Google Drive, OneDrive and Dropbox aren't a good idea for the
master copy: they're small and they're at the wrong end of the internet.
They're also vulnerable to 'ransom risks': if they announce pricing for next
month will be $500pm, what do you do?

Another risk is site failu fotopic.net went under and it turned out the
bandwidth bill for sucking all the photos off it was beyond the means of the
company that had just called in the receiver because they had no money.
Likewise Code Spaces got hacked and the attacker just deleted all their
files - there was nothing left. The message from this is that if a cloud
provider goes under, it's simply gone, you don't get your data back.

However they could be useful for transit. For instance, git is a nice
protocol because every working copy contains a full copy of the history.
That means I don't care if github goes under - I have everything locally.
So github is simply a nice website for publishing the sources, and if github
went away it wouldn't be much effort to republish elsewhere.
If the files are encrypted you also don't need to worry about privacy
(though metadata is still an issue, and always the devil is in the key
management).

So I could see how something like this could work - Dropbox et al are just a
transit provider for sharing stuff back to base. This also means they're a
commodity conduit - if Dropbox are charging £silly, just switch to OneDrive
(or whoever). The one thing I don't see this doing is sending a request to
the NAS saying 'please put file XXX in Dropbox so I can see it' - maybe
there's a way round that. Or maybe you just say 'you can only view the
photos when you're at home'.


On the wider philosophical question, I found it depressing to read of an app
for sending encrypted SMS where the developers are tightly wedded to the
Google Play infrastructure, while seemingly not realising the tensions
between these two directions:
https://github.com/WhisperSystems/TextSecure/issues/127
As that thread demonstrates the issue is somewhat complicated, but I'm not
surprised that many mobile developers don't understand the risks.
(That app developer has softened their position somewhat in later posts)

Theo

On Wed, 04 Mar 2015 13:14:56 +0000, Tim Watts wrote:

SFTP so far offers the simplest solution but I can find no nice clients
that offer local copies (caching) so it is still somewhat half arsed.
And i suspect the basic nature of the protocol is it would be hard work
to make caching multiple client copies work.

No caching, but the well-known gftp FTP client also supports SFTP. IME it
gives a seamless, trouble-free experience with either protocol.


--
martin@ | Martin Gregorie
gregorie. | Essex, UK
org |