


The Bibledit-Gtk notes become personal after the consultation notes on Bibleditg-Web can be operated by email.
To break the link.
To see what to import still.
To look for an easy way to import these notes, 
but better is to leave this and do it by hand since it should not occur often.
































De site zendt elke dag emails met veranderingen in de Bijbeltekst. 
Een knop voor 'send now', net als bij backup, en dan elke dag daarna om dezelfde tijd.






Gtk The backup routine keeps going many times.







the Ubuntu package git-daemon-run may be a useful way for
you to set up a git server that is permanent and restarts itself
appropriately after server reboots.  







To edit the note contents, for the Manager.
Import from Bibledit-Gtk: Do not import logbook entries.







Git operations as follows: No longer just deleting all stuff as we do now.
But we only delete data when a chapter gets deleted.
To add this 'delete' operation for all database operations.
Thus the shared dictionary remains.
Thus the status data remains too.
If a book gets deleted, then the whole directory gets delete for that book.





Notes date selection mechanism: 
Those which have not had any activity on them for, e.g., a week or more, or a month or more.







Upgrade Bibledit-Web through the web. 
Download tarball. 
It installs from that file.
Information: how to create a tarball from the repository. This may link to the developer help pages.
I believe that this is too much of the good, and should not be allowed to be done.
To remove all code for it.









When a note is deleted from the system, it still it in the git repository. It should be deleted from there as well.
Same applies to a chapter of the Bible. It works thus: Once the data is send/received, then it goes through the whole
git file system, and checks all notes or all chapters. Whichever is not in the database will get deleted. Thus
the git repository gets updated.










The 'Create new note' link is not visible when working or editing a note. Perhaps the idea can be to have 
that link visible always. It is easier to work as well.







The sorting of the notes is not right yet. It should sort chapters and verses numerically.








Sending out changes to the text.
Let say, one hour after the last change in a chapter it will send out this chapter, along with any other
chapters, to the subscribed users.







Export the USFM data to the .exp file for the Online Bible.
See OLB Help/Table Of Contents/How To/Create Your Own Bible.













To automatically import translated templates from launchpad.
bzr init
bzr pull lp:~teusbenschop/bibledit/bibledit-pot
To import the translation templates into the source.








The site needs to have the concept of Desktops. Users can create store and erase these. 
E.g. a desktop for editing, one for merging, etc.
The desktops contain any combination of screens or tools that the user can use.
There are standard Desktops included in the default application. The user cannot modify these.
When editing desktops the user can choose in which menu it will be displayed, and it will be displayed there.






A user can subscribe to changes in a Bible.
These changes preferably are given only once an hour or so and the 
email will contain all changes assembled into one.
There will be links with the changes that allow the user to go straight to the passage.
Changes are given as in Bibledit-Gtk, where differences are shown at the character level.













Messaging between applications and server for e.g. verse reference, and many other things, 
e.g. lists of verses to be sent to the references window.











The text editor needs a link that sets the default / focused bible, so that the navigator takes that bible.
But for as long as the navigator is not yet separated, then this is not needed, but we should prepare for it in the code.


















If a Bible translation work is collaborated in public domain, it could be too hard come up with one
translation for every verse. It is too bad to split the team and make a copy of the web application 
every time they can not come up with a common translation. So I am assuming that branching at verse level
should be supported and the final outputs (like whole OSIS, let me call it "a distribution") should also be plural. 
And for each distribution should choose branched verses to take which version.







Just a thought here, but I think it might be good to have the option for administrators to chose which rights go to whom. 
E.g. I'm not sure a new translation really should be immediately available to anyone ("visitor"). 
It may invite premature comments or criticism that would slow the process down. 
Or perhaps visitors could be prevented entirely until a certain phase when the administrator chooses
to disclose translation content.












The book names are not localized as we do in bibledit-gtk, but localization uses the existing mechanism.
It works through gettext. For that reason a php file needs to be created that contains all existing English books.
This file is to be generated somehow by the system admin, and this file can be stored among the source code.
Thus the gettext system will be filled with that data, and can get translated.
If somebody wants the localized Bible names, but does not translate the whole site, it suffices to only translate 
the relevant bits in the po file.



















Pay attention to internationalization.
We need to get gettext working, and so need to have accounts where the user can set his language.
The administrator sets the default language for the site for guests and before login.


On Ubuntu for extra locales, install packages like "language-pack-*". After installing a new pack, run "locale-gen" and restart Apache (sudo apache2ctl restart)
to make this new locale to take effect. This should go in the installation manual.


To go through the various php functions called and read the comments, and learn from it.
setlocale(LC_ALL, ""); Takes the locale from the environment.
Before setting the locale, the php page runs 'locale -a' to see the available locales on the system.
Then, it needs to check whether bibledit provides this translation, and advise the user appropriately.

A little function to test available locales on a sytem :
<?php
function list_system_locales(){
    ob_start();
    system('locale -a'); 
    $str = ob_get_contents();
    ob_end_clean();
    return split("\\n", trim($str));
}

$locale = "fr_FR.UTF8";
$locales = list_system_locales();

if(in_array($locale, $locales)){
        echo "yes yes yes....";
}else{
        echo "no no no.......";
}
?>


It may use the available locales for dates and number representation as are on the system, and use our own locales 
as come with the application.
These two catalogs are different.





When setting translation domains in the preferences, do a check on the functions' return values if it worked out well.




The administrator can leave the locale empty, in which case the system locale will be taken. 
This should be the default for local installations.





The site has one default translation, set by the site administrator. Visitors get to see this default translation.
















To have a module that creates a new GoBible each hour if there were changes. The user also can request to make a module "now", 
which will be scheduled for the next minute. If there were no changes, it indicates this as well. There is also a very simple page for 
access from cell phones.





People with an account can subscribe to changes in the text.












The visitor to the site can see the default project, its text and comments. 
But to make comments, one has to get an account.
Those with an account have vastly improved opportonities on the site.












Windows: http://www.wampserver.com/
Or: http://www.apachefriends.org/en/xampp-windows.html
Easiest: EasyPHP.







Instead of using XeTeX, we could also look at CSS2 with paged media for printing, and screen media for the screen.








Where help is needed: 
Would need to write the software for the project under the GPL, stated in each source file.
Could fully test current source on Windows, describe how to install it, 
and create a Windows installer and describe how to do that through open source tools. 
Same for Macintosh. 
Would communicate through bibledit-development. 
Would write documentation on the Wiki. 
Could write manuals for how to work with Bibledit-Web. 
Could make a start with an issues tracker that is compatible with Bibledit-Gtk's project notes. 
He could enable internationalization and localization with a language setting in the user preferences, 
and a language setting for the site administrator for the site's language if the user has not yet logged in.












Mails to be sent out upon change in text. Can work now already, since there can be synchronisation 
with git repository. User may subscribe to any changes, or to changes per chapter.








The book names should be translatable. So it needs a list of all bible books to be included for gettext translation,
in .php format.








There can be a Notification setting that the user can set up which Bible or Bibles are included with the 
notifications on a note.














The Consultation Notes verse list, if the Verse List shows, 
then there is a link in the consultation notes [transfer to verses list],
which does what it says. This link is only shown if the verse list shows. If it does not show, then the link is not there.





















Producing a sword module for Katana.
Use usfm2osis.pl
If all .usfm files are in subdirectory ndebele, do this:
perl usfm2osis.pl Bible ndebele/*.usfm
This produces a file Bible.osis.xml.
Then make it a Sword module:
osis2mod . Bible.osis.xml
To write it to the proper place:
osis2mod ~/.sword/modules/texts/bibledit/ndebele Bible.osis.xml
Teus,
osis2mod is conservative with file permissions.  Be sure the files have
proper permissions before you deploy.
I only mention this because I often forget to grant read access to files
before deploying a module for testing and the result is only
chapter/verse numbers.















> Is there any news about packing bibledit-web? 

I started on this today.  I soon encountered something I am not sure how
to handle -- bibledit-web seems to expect a working and configured mysql
server at *build* time.

This seems unusual; I can understand needing a MySQL server at run time,
but not at build time.  I should be able to compile and install (and
package) bibletime-web without a MySQL server, shouldn't I?

The Debian and Ubuntu builder machines do not provide MySQL servers
during the build process.  Even if they did, those might well be
different (and configured differently) from the mysql server seen by the
app when the package is installed on a user's machine at some later point.

I would expect that the build process just builds and installs the
application; there would usually also be a default configuration file
which can be edited to specify where the database server is, username
and password, etc.  In the Debian/Ubuntu packaging world, initial
settings for these would be done using debconf, so the user gets a
dialog asking for them at install time.

I hope the database server/name/username/password information is not
being hardcoded into the application itself during the build?  Surely
the right place for that kind of info is a separate configuration file?

I could try to patch out the checks for the MySQL server done by your
configure script... but then I feel I'm basically working 'against' the
developers intentions instead of with them!  You (presumably) put those
checks in there at build time for a reason... I just don't know what the
reason is.

phpmysqladmin (your suggested example existing packaged web app) does
not do this kind of thing during its build process.

Incidentally, longer term, you might want to consider using a database
abstraction layer (PEAR MDB2 for example) so that bibledit-web can use
Postgresql or other database back ends, not just MySQL.

Thoughts, ideas and suggestions welcomed :)

More comments and ideas related to packaging bibledit-web:

CONFIGURING TOO EARLY:

Looking further at the autotools configuration done in bibletime-web
0.2, other things that need to be settable per-installation, not at
build time, are apparently also being configured and defined at build
time.  These include the database server location, database admin user,
database admin password, database name, database user, database
password, bibledit admin user and password, and the location of the
webroot of the web server.  Some of these seem to be handled in
database/credentials.php and this is created by setup.php, but in that
case, why is the checking of MySQL at build time being done?

Why the code needs the mysql admin username and pw is a question in
itself -- wouldn't it be more secure to prompt for these when needed,
which hopefully is just for a one-time initial database setup, done at
(package) install time, not build time?

Since (at least in theory) one could set up multiple instances of
bibledit-web on a single server, each with their own database and each
with their own apache virtualhost, doing too much at build time seems to
me to reduce overall flexibility and capability.

Unless I am misunderstanding what the code does, I don't see how this
software can readily be packaged until many or all of these things are
moved from being defined and checked at build time to being settable in
a configuration file accessed (and checked, if appropriate) at run time.

DATABASE CREATION:

I think the setup.php code that creates the bibledit-web database
probably will need to be made usable as a script to be run at package
installation time, or maybe at first web login to bibledit-web.  Asking
users to point a browser at a URL just to do something non-interactive
seems unfriendly.  The web server needing to be able to delete the
bibledit-web php files (auto-deleting setup.php after running it) looks
like a security weakness; only the server admin should be able to edit
or delete those files, once installed.

LACK OF $(DESTDIR):

I've also found what seems to be the same kind of issue I found earlier
in the bibledit-gtk Makefiles, where an install-data-local target does
not use $(DESTDIR).  This time the issue is in web/Makefile.am where the
four lines under the install-data-local: target need $(DESTDIR) before
$(WEBROOT).

Fixing that still leaves it installing directly into the root of the
default host, /var/www/, which is not allowed, but that's a different
issue, more related to the "configuring too early" problem noted above.

SECURITY:

There is too much chmod -R 0777 going on (both in the Makefiles and then
also in some PHP code) than can be considered safe or secure for an
Internet-facing web server.  World writable files and directories are
almost never needed, and almost always a security risk.  Why would every
user on the web server machine need full access to all of these
bibledit-web files and directories?  Installed on a shared web hosting
server, this could quite easily allow other customers on that server to
do anything they like with your site, including add their own code, run
spambots as you, hand out malware to every unsuspecting user who uses
the bibledit-web site, etc.

I'm not sure that suggesting or assuming a MySQL root pw of "root" is
wise, either :)  If you are going to do setup from PHP, why not have
setup.php prompt for all the database info (host, dbname, adminuser,
adminpw, username, pw) and write some of it (hopefully not the
adminuser/adminpw!) to a config file?  It can do some of that already.

All uses of system() and exec() should probably specify a full path, so
users can't persuade your code to execute *their* chmod instead of the
system one, for instance.

I'd suggest you try to do a detailed security review as soon as
practical.  It seems odd to be using 1024bit ssh keypairs, but also
doing things like "chmod 0777" and assuming a dba password of "root".

INSTALLATION HELP:

The help button points to help/001ubuntu1004server.php but this is
specifically for 0.1, and the codebase is 0.2.  Are there any
differences?  Also, this file says in part:

  If the site is "site.com", then this will be something like:
  http://site.org/bibledit/setup.php

You can use either site.org or site.com, but they need to match :)
Using example.com is common for this kind of text, incidentally.

EMBEDDED LIBRARIES:

bibledit-web seems to have included a copy of the smarty library within
its codebase.  This is not allowed when packaging.  I'll need to change
it to use the system-supplied already-packaged version(s) of smarty and
smarty-gettext (does it actually use smarty-gettext, or just smarty?).

SUMMARY:

As far as I can see, the bibledit-web 0.2 codebase pretty much requires
that it be built from source, locally for each individual installation.
 That is the exact opposite of what packaged software does (build it
once, then distribute the resulting compiled binary package around the
world) :)  Since bibledit-gtk doesn't require this
unique-per-installation build, I'm not sure why bibledit-web does, or
what the benefit of this approach is.  I rather suspect that in trying
to make things "friendly" for people installing it from source, the
software has been made hard or impossible to install as a package.

Hoping we can find a good approach to packaging bibledit-web,




I now have a hacked-up bibledit-web 0.2 package that more-or-less does
what the source install does, and installs on a test virtual server.
But it's much too rough to release, it breaks many rules, and won't even
build in a PPA, never mind in the official builders.

A few more comments and questions arising from this:

DATABASE CREATION vs DATABASE USE:

I am rather alarmed to find that the bibledit-web code seems to make no
distinction between the database user/pw needed to create the bibledit
database and its tables, at install time, and the one used at runtime to
work with those tables.  Am I understanding this correctly?

By default, this leads to the bibledit-web database code using the
all-powerful DBA user (root) at run-time, and storing that username and
*password* in plain text in a configuration file.  Worse still, the
default password is "root", and (even worse!) that configuration file is
set to mode 0777, so absolutely any user on the server can read it, and
then use it to do absolutely anything they want with any MySQL database
on that database server.

I believe this is a significant security issue.  On my web servers
(servers I admin, not servers I own!), a single MySQL server instance
can contain databases for 100-200 independently owned web sites... the
bibledit-web 0.2 installation in effect gives *every* user on the server
full access to *every* *single* *one* of those databases! (I didn't
install bibledit-web on a production server, so no harm done, I'm just
pointing out how bad this would be in production use!).

I think the right solution is for the code to distinguish very clearly
between (a) the DBA username and pw, and (b) the runtime
bibletime-specific user and pw.  (a) is prompted for and used only once,
at install time, to create the bibledit database and the runtime user
and pw. It is never stored anywhere on disk.  (b) is then used to create
the tables, and then to access the information in database, by
bibledit-web at runtime.  It is stored in a config file (readable only
by the user running the bibledit virtualhost, but not by others!).

I have not checked whether this issue is fixed in the latest git
development code; I probably should :)

LIBRARIES:

Is there a list somewhere of what libraries are needed by bibledit-web,
or are incorporated as copies (or modified copies) within it?  So far I
have found:

 * jquery
 * jwysiwyg (a plugin for jquery)
 * smarty
 * a modified copy of part of zendframework

Are there any more?  At least the non-modified ones have to be packaged
separately, if packaged versions of them don't already exist.  And I'll
have to look at the modified ZendFramework code and see how best to
proceed there.  The INSTALL file might be a reasonable place to put this
kind of information.
Ubuntu has libzend-framework-php





> When starting on Bibledit-Web I had in mind that the tarball would
> be installable on Linux systems, but would also be uploadable to a
> server through ftp and install itself there. But these seem to be two
> different things. For example, inclusion of smarty(-gettext) is
> necessary when uploading through ftp to a server the user only has
> access to that way.

Well, logically installing to a shared account on a remote web server
could be done by obtaining each of the tarballs for bibledit-web and for
each library that it needs (which is are not already installed on that
remote web server), untarring and configuring each of these, and then
uploading the resulting set of files using FTP (or, preferably, using
SFTP so you are not passing your database and admin passwords around
over the Internet in the clear; no-one who cares about security should
be using FTP any more).

You could perhaps look at creating a "just bibledit-web" tarball which
includes an installation script that downloads the libraries it needs
(using wget or similar) and untars and configures them for bibledit-web
use.  Then the person doing this kind of install does:

 * Download bibledit-web tarball to a local Linux PC
 * Unpack bibledit-web tarball locally
 * Run installer script which downloads/configures libraries
 * Upload the resulting tree of files to the web server

Those installing to a machine they have full control over would instead
install the necessary libraries as packages, and configure bibledit-web
to use those copies of the libraries (or, one day, install a
bibledit-web package which would depend on the library packages and
configure itself to use them!).

> There is a clash between this practice and packaging things. When I 
> asked for a bibledit-web package, I had not realized the implications
> of that.

Yes, bundling everything into one tree of files is convenient for FTP
uploads, but unhelpful for packaging.  Doing work to avoid using
included "convenience copies" of libraries when packaging is fairly common.

Also, if this is the design goal, then I think checking for MySQL and
needing the root pw at configure time is unhelpful -- the local Linux PC
may not even have MySQL at all.

Incidentally, I'm a bit surprised that your users are successfully
running PHP code that uses system() and exec() to run Unix commands such
as git and ssh-keygen on machines they only have a shared web hosting
account on.  Good shared web hosting providers tend to use the
disable_functions entry in php.ini to disable "dangerous" PHP functions,
often including system() and exec().

> The tarball as it is now tries to make it easy to install the
> software from source. Whether it succeeds in that is a different
> matter.

I think it depends greatly on the server setup; it seems it is fairly
easy to install bibledit-web 0.2 from source if it is the ONLY web
software and ONLY MySQL-using software on the machine, and if Apache
ONLY serves up one (default) web site.  So for a quick test install on a
freshly installed Ubuntu desktop, it's relatively easy.  But for
installing on a production Ubuntu server that already uses MySQL for
other things and has multiple apache virtualhosts in place, it's not
going to be easy at all.

> The problems that you mention still need to be addressed. I will do so
> when time permits. Clearly, the software as it is, is not yet fit for
> packaging.

I think that's probably true, unfortunately.  I'd suggest dealing with
the "chmod 0777" and "root MySQL password stored in config file" issues
quickly if you can, since they are not packaging specific, and seem to
me to be exploitable security issues.  The rest... can wait :)


