Horizon 2.0 docs checklist

[dalanmiller]

With the impending release of Horizon 2.0 we need to make sure our bases are covered in the docs:

Please also review @deontologician + @Tryneus

• Make it clear engine.io is no longer used
• .hz dir now may contain config.toml, schema.toml, and secrets.toml hz schema streamlining  horizon#679
• hz set-schema has evolved to hz schema apply rethinkdb/horizon@0e9e294
• hz get-schema has evolved to hz schema save hz schema streamlining  horizon#679
• Writes will now attempt to retry on Horizon server Write retries with timeout (server side) horizon#636
• Aggregation queries Implement aggregated queries horizon#329
• _r is now available on Horizon server object (so you don't have to reimport rethinkdb Add _r to Horizon Server object. horizon#659
• Auth0 is now available for OAuth Auth0 horizon#601
• Remove references to the client logging functions (if any) .enableLogging
• Document rdb_timeout, rdb_user and rdb_password options Security options for rethinkdb horizon#507

Additional things before Horizon 2.0 beta release is ready that are currently incomplete:

• Instruct users that collections cannot start with hz_, this will result in an error (limitations page)
• hz migrate command
• hz schema apply will be able to read schema.toml files from versions <= 1.1.3
• hz serve will automatically recognize <= 1.13 db tables and index names and error out
• hz schema save will now keep the older .hz/schema.toml file if it already exists and apply a datetimestamp to the end of it.
• Index declaration format in schema.toml will change.

[deontologician]

Also @Tryneus is getting rid of the horizon_internal table, and indexes will be named differently: rethinkdb/horizon#650

[chipotle]

When you say "make it clear engine.io is no longer used," what do you need to have mentioned? (AFAICT we don't mention "engine.io" anywhere in the current set of docs.)

[dalanmiller]

When you say "make it clear engine.io is no longer used," what do you need to have mentioned? (AFAICT we don't mention "engine.io" anywhere in the current set of docs.)

That's all I wanted to accomplish there @chipotle, just wanted to make sure there weren't any loose mentions hanging around.

[deontologician]

I added another bullet for checking for references to client logging functions. I doubt they were documented, but they were removed so we should make sure

[dalanmiller]

Where are the aggregation queries documented? (I wish Github told us who checked which checkmarks)

[chipotle]

They're in the commit that closes #99, which is merged into the horizon-2.0 docs branch. I'll do a PR for that whole branch to close this ticket.

For rethinkdb/horizon#636 is there a way for the user to set the write retry timeout?

[dalanmiller]

@Tryneus ^^

[Tryneus]

@chipotle - currently there is no client support for it, it's just an option available in the protocol.

[chipotle]

Okay, so is there a timeout enabled in Horizon 2.0? I just want to know if I should be deleting the "Horizon will not automatically retry writes that have version conflicts" line from the limitations document, and adding something to the FAQ saying "Horizon will automatically retry writes for ___ seconds." :)

[chipotle]

With respect to "Remove references to the client logging functions (if any)," does that mean that the debug option needs to be removed? (There's nothing in the docs about .enableLogging as a method.)

[deontologician]

@chipotle the debug option in the cli still works, and any logging in the server still works. You used to be able to do Horizon.enableLogging() to get console messages when sending or receiving on the client. It was unlikely anything was documented I just wanted to make sure

[intellix]

This was fixed/merged: rethinkdb/horizon#498. Is there any migration info for new table names required for the docs?

[dalanmiller]

Nice work all 👍 🚢

[dalanmiller]

One more @chipotle:

• Instruct users that collections cannot start with hz_, this will result in an error (limitations page)

[dalanmiller]

@danielmewes added more to the check list from our discussion ^^

[danielmewes]

rethinkdb/horizon#704 has a list of the corresponding issues for the missing pieces in the main Horizon tracker.

[chipotle]

A couple questions about some of those pieces:

• If hz schema apply can read a pre-1.1.3 schema, does it do any conversion, or does that only happen when you use hz schema save?
• What does hz migrate actually do? (i.e., does it work on database tables, and is it what you run to fix the problem with hz serve "erroring out" if Horizon 2.0 tries to serve the old format?)

[danielmewes]

@chipotle
hz schema apply does not convert the schema file itself. If you want to get a new schema file that uses the current format, you have to apply it first, and then use hz schema save to export the schema in the new format.
The only difference for the schema file is how indexes are defined (currently a single string that contains all the indexed fields separated by underscores, starting with 2.0 a structure with multiple fields - details pending).

hz migrate works on the Horizon metadata that's stored in RethinkDB. If you have an existing data set for your Horizon application in RethinkDB, it will convert the corresponding metadata, so you can keep using the same RethinkDB database and don't lose any data.

For the upgrade from 1.1 to 2.0 specifically, it will:

• rename secondary indexes on the tables that store the collections to adhere to a new naming scheme
• delete the <project name>_internal database and transfer the metadata stored in there into a set of new tables within the <project name> database

It is indeed what you should run to get over the error from hz serve when running on a database with old metadata.

[danielmewes]

One more thing, which I think we forgot about:
There are new rdb_timeout, rdb_user and rdb_password options for the config.toml file, the CLI commands, and the Horizon server constructor. See the first post in rethinkdb/horizon#507 .

[deontologician]

• make-token now doesn't connect to rethinkdb. It can create a token for any user id, whether it exists or not

[danielmewes]

@chipotle The new format for specifying an index in the schema.toml file for hz schema apply:
rethinkdb/horizon#706 (comment)

[intellix]

An example with syntax similar to how the current documentation (http://horizon.io/docs/permissions/#configuring) tells you how to create indexes:

[collections.markets]
indexes = [
  'hz_["event_id"]',
  'hz_["event_id", "name"]'
]

first is a single index, second is a composite index

[danielmewes]

@intellix Thanks for the example. Your example is essentially using the old schema.toml syntax for defining indexes (where each index is simply described by a string), though with the new Horizon 2.0 naming scheme.
We are actually making the index definition more structured, see the example I linked above: rethinkdb/horizon#706 (comment)

I think your syntax will no longer work for the final Horizon 2.0 hz schema apply tool actually. If will be able to read Horizon 1.x index definitions for backwards compatibility, but since your definition is a mixture of Horizon 1.x specifications and the Horizon 2.x index naming scheme, I think the tool will probably get confused and might reject these.

[doefler]

Hi I just did npm install -g [email protected]
Now I am trying to run hz migrate which is not working for me.
I am running the command from the my project folder.

I have tried a lot of variations which haven't worked for me. Can you point out what I am doing wrong:

hz migrate
hz migrate myhorizonapp
hz migrate --project-name myhorizonapp
hz migrate -n myhorizonapp
hz migrate -n "myhorizonapp"
hz migrate "myhorizonapp"
hz migrate ./
hz migrate .hz/config.toml
hz migrate .hz/
hz migrate .

Errors I got:

/Users/username/.nvm/versions/node/v4.2.1/lib/node_modules/horizon/src/migrate.js:119
throw new Error('No project_name given');
^

/Users/username/.nvm/versions/node/v4.2.1/lib/node_modules/horizon/node_modules/toml/lib/parser.js:3946
throw peg$buildException(null, peg$maxFailExpected, peg$maxFailPos);
^
SyntaxError: Expected "#", "E", "\n", "\r", "_", "e", [ \t] or [0-9] but "." found.

Thank you.

[danielmewes]

@doefler This issue is about the documentation for Horizon 2.0. The issue you're running into looks like an actual issue with the hz migrate command. Would you mind opening a new issue about this in https://github.com/rethinkdb/horizon/issues ?

[doefler]

@danielmewes thank you for responding.
I will open a new issue right away.
Since I am not able to run ´hz serve´ without migrating, do you have any ideas on how I can work around it until the migrate tool works?

[deontologician]

The --config option is no longer accepted on any cli commands. Your config files must reside in ${project_dir}/.hz (so secrets.toml, config.toml and schema.toml)