- Find var doc & examples when you know exactly what you’re looking for
- optimize for speed
- Find things when you don’t know what you’re looking for
- see alsos
- namespace nav
- quickref / area (string manipulation)
- concept (I want to do x)
- Mobile: main use case as linked-to from blog posts and such
- think about removing all the extra parts for mobile
- Site experience
- Must be robust and snappy. Perception of a language is often tied to factors out of that language’s control, e.g. network congestion, non-optimized css transitions, etc. A quick and responsive experience is a top goal of the site.
- How-tos on web app | desktop | mobile
- Libraries?
- Basic copy for core namespace pages
- Remove non-core see alsos
- numbers sanity check
- Fix horizontal scroll issue on intro page
- Fix wrapping on code pres
- Mobile Fixes
- Fix scroll-to on expand in firefox
- Fix original author not being attributed correctly
- Move over from old db
- favicon
- check indexes
- Redirect old urls to new
- Claiming old user accounts
- Add user page
- Redirect old /v/# urls
- Responsive ns page
- Notes CRUD
- See alsos CRUD
- page titles (var info, etc)
- Fix function counts on autocomplete
- Make code text sizing responsive
- Exception reporting
- Editing examples
- Refactor examples and see alsos lookup for autocomplete results
- Add example widget
- Core-specific namespace page
- Deleting examples
- Switch to avatar url instead of email
- Add libraries page
- Show no results state for search
- Add number of examples to autocomplete
- Loading state on search inputs should show immediately on input
- Var page navigation
- Mobile slide-out menu
- Redirect old versioned var urls
- Rename comments -> notes
- Null-state for top contribs
- Fix login redirects to current page
- Move over comments
- UPDATE / REMOVE ROBOTS.TXT
- Check db indexes
- Move example submission styleguide over
- Landing search feedback
- Expand and redirect shortened namespaces
- Handle abbreviated namespaces `c.c` -> `clojure.core`
- Landing search feedback
- User accounts page
- Fingerprint & caching on assets
- CLJS refactor
- Better throttling for search widgets
- Staging site header
- Source link on var page
- See alsos rendering
- Fix footer on mobile
- Update clojure tutorials
- Encode clojure names for urls
- Disable input until js loads
- Example history page
- Footer
- GH / Twitter links in footer
- Switch sass -> less
- Quickref attribution
- Consistent h3 styling on quickref page
- Float quickref TOC
- Quick lookup in top bar
- Namespace sidebar on var page
- Use ssl for external resources
- Landing page search widget
- Loading indicator on search widget
- Add recently viewed vars to sidebar
- Animate scroll-to links
- Jobs
- ClojureScript / Clojure.NET support
- URL shortener howto
- Investigate bitbucket login
- Quickref update
- Supported libs blurb
- Split-screen example editing
- Admin interface
- What goes here?
- Example history page
- Backups
- Clean data export for dev distribution
- optimistic network
- Organize ns decls
- Auto load namespaces on file changes
- Document live reload
:namespaces
- name
:examples
- var.name
- var.ns
- var.library-url
- deleted-at
- author.login
- author.account-source
- editors.login
- editors.account-source
:see-alsos
- from-var.ns
- from-var.name
- from-var.library-url
- to-var.ns
- to-var.name
- to-var.library-url
- author.login
- author.account-source
:notes
- var.ns
- var.name
- var.library-url
- account.login
- account.account-source
:legacy-var-redirects
- function-id
- editor.login
- editor.account-source
:example-histories
- example-id
:users
- login
- account-source
:migrate-users
- migration-key
- Idempotency
- Users
- Construct avatar url
- Vars
- Import on startup
- Jobs overview
Let’s see if we can declaratively define our endpoints:
- routing – server & client
- db querying – server
- input coercion / transformation – server
- validation – server & client
- data querying / storage – server
- error handling – server & client
- failed validations
- 422 :general-error, :prop-errors
- failed validations
- response transformation – server
- authorizaton – server
- Server
- routing
- db querying
- db storage
- validation
- error handling
- response transformation
- authorization
- Client
- routing
- validation
- failed validation reporting
It would be nice if both the client call functions and endpoint generation could be done off of the same data.
Single-field validation, form submit -> routing -> input coercion
(client-call schema context success error)
It’s unfortunate that you have to visit 3 different namespaces to add a single endpoint, this needs to be fixed. Part of the reason right now is that the schema is cross-platform and the endpoints are not. Another part is that you have to explicitly mount endpoints in `clojuredocs.server`. See: https://www.youtube.com/watch?v=_oj0gfSRLm0
All this is fixable I think.
Actually four places, also the front-end code that uses that endpoint.
- clojuredocs.api.schemas – schemas, cross env
- clojuredocs.api.server (clj)
- clojuredocs.api.client (cljs)
(api/request api/get-examples payload on-success on-error)
(api/response api/get-examples handler)
(server/endpoint get-examples handler)
+clj (defn get-examples [r])
+cljs (defn get-examples [opts success error]
- Config
- Init
- Client API
- Page Rendering
- Comps
- Data Access
- Server Comm
- Widgets
- Cross components
- Routing?
Used to house all page rendering logic. Adding a page? The route / logic should probably go in pages.clj. If the logic for rendering your page needs a bunch of supporting functions, or if there are many pages that should all be grouped logically, consider adding a namespace to the pages subdir (see search feedback routes, for example). Routes should only be added to pages.clj unless you really, really need to handle routing in your namespace.
- Use Cases
- View example – Show author / editors in example meta, and latest edit body and last updated timestamp
ClojureDocs v1 used OpenID, which is being phased out pretty much anywhere (oh well). So we’ve got a bunch of user accounts from the old version of CD that have to be migrated over.
Personally, I don’t really want to maintain a user identity that’s specific to ClojureDocs. I like the idea of using GitHub auth (oauth2) initially and adding on other providers. Just have to handle the case of login collisions from different providers.
- Styleguide is a good stage for isolation
- Widgets should work both in context and in isolation
- Using channels to isolate widgets
- How does this work with nested widgets?
Is an appropriate shape of a widget the:
- state
- no external calls
- state is not modified in-widget?
That last one is interesting, lets visit that. Another way to state that question is: should all app state modification be done out of widget?
Probably not, error handling becomes to arduous. So then isolation becomes a matter of configuration.
- Should be able to put the widget in any of it’s states in a straightforward way (passing data). Channels hurt here.
- Composibility of widgets is important, channels add another dimension when thinking about composibility.
- Right now state transitions are hard to follow, alot of it is spread across the namespace. What’s the cause of that?
- Should event handlers just be about putting stuff onto a queue?
- Maybe localizing all mutation to the widget itself isn’t a bad idea, as opposed to channels escaping the widget. This begs the question of how inter-widget communication should work (refs?).
Results are looking good so far, essentially all state manipulation happens in the same place, which is a good thing. There’s still the question as to whether this could be accomplished by just moving code around. Even if not, is it worth the overhad managing channels?
(defn event-loop [state text-chan cancel-chan]
(go
(loop []
(prn @state)
(let [[v ch] (async/alts! [text-chan cancel-chan] :priority)]
(condp = ch
text-chan (do
(swap! state assoc :text v)
(recur))
cancel-chan (prn "CANCEL"))))))
(let [t (async/chan 10)
c (async/chan 10)]
(event-loop (atom) t c)
(go
(>! t "foo")
(<! (timeout 100))
(>! t "bar"))
#_(go
(>! c "hello")
(<! (timeout 100))
(>! c "world")
(<! (timeout 1000))
(>! c "the quick")))
It really seems like overkill instead of manipulating state directly in the event handler, but let’s try it.
Ok, hit a road block. An example doesn’t know how to remove itself from the page, so there needs to be some communication outside that handles this.
Two problems:
- How is this communication structured? A channel that the widge
exposes? A channel that is passed to the widget to put deletes on?
An event that’s fired? The widget is passed the shared structure
and it removes the example from the list of examples? A flag on
the example that prevents it from being rendered?
- If it’s a channel passed into the widget, how is that passed through multiple layers of widgets, if multiple layers exist?
- How to identify the example to be removed?
(defn wire-meta-behavior [owner example report-delete-chan]
(let [delete-ch (chan)]
(go-loop []
(when-let [delete-state (<! delete-ch)]
(condp = delete-state
:do-delete
(let [res (<! (delete-example example))]
(if (= 200 (:status res))
(!> report-delete-chan example)
(om/set-state! owner :delete-state :error)))
(om/set-state! owner :delete-state delete-state))
(recur)))
{:delete-ch delete-ch}))
I’m not a huge fan of how the event handlers are bound by specifying the function to run in the element attributes.
Ok, it’s a little better, but still coupled. I’ve been thinking about having a single channel / pair of channels be the communication interface to a widget, and how state is manipulated, so information flows:
app state -> events -> channels -> app state
Server comm should probably be done at the top level, or as high-up as possible
Widgets interact with state either through putting messages on provided channels, or manipulating the state they have in scope (cursors).
component / page / container / world / widget / loop / link / module / mod
Mod – app state
Included declaratively
Lifecycle
- People will have a harder time finding what they’re looking for
- Site will be slower
- Site will have more (any) downtime
- Contributing will be difficult because the codebase is a mess
<2014-09-13 Sat 21:48>
Removed syntax highlighter in favor of the one cribbed from Reagent. Feels way more performant, and I can remove syntaxhighlighter & supporting files from the frontend.
458K pre, 433K post :D
<2014-09-19 Fri 16:40>
Feels like I’m getting close. The to do list is down to about 5 items, but I’m sure I’ll add more before launch. Got the DNS servers switched over from 1and1 to Route 53. I really need to get that domain reg switched over to namecheap. 1and1 is such a POS.
Copy is the next big thing, I’ll do on or two passes, but there are much better writers in the community that can help out.
Post-launch: autocomplete is way, way to slow. Was getting ~1-2 seconds turn-around on my mobile tether, prob get around 250ms on cable. That’s still to slow I think, we need to try shiping all the vars down and do the search on the client next. It’s not that much, ~1000 vars clocks in around:
1000 vars * (50B name + 200B docstring + 100B fudge) = 350K extras
Less w/ gzip
I’m guessing that extra 350K is worth it if cachable.
Getting close to a release, just need to get some help banging on the thing. Also working on persisting search state:
There’s a ton of stuff once you dig in.
- back / forward through queries
- restoring state on back
- store autocomplete results in local storage to be able to instantly populate on back nav
The search stuff needs a good refactor anyway, the separation of concerns and encapsulaton is poor
Getting everything updated to 1.8, which was released months ago. You know, 1.9 alpha 10 is out, I should update to that while I’m at it.
Also I’ve just noticed that the delete example link isn’t working.
Add / update works.
Delete on see-alsos work. Delete on notes work.
I should compile for prod locally to double check on this release.
Added display of the current clojure version to the homepage for desktop / mobile. Really love the mobile slide-out menu, that thing has really held up over time.
https://cl.ly/2S100u241x2y/Screen%20Shot%202016-08-20%20at%204.48.04%20PM.png
https://cl.ly/3m3E3f3l1F2o/Screen%20Shot%202016-08-20%20at%204.48.36%20PM.png
Also checked 1.9.0-alpha10, seems to be good to go so the next version bump will be nice and easy.
TODO
- searc.cljs – update :ac-text (om/update! app :ac-text text)
Continuing the reagent transition. Still in the middle of the yak shave to get the app lifecycle stuff implemented – js reloading (that reminds me I need to convert the less over to garden, and to do that I need to get reup in).
I’m a little burned out on getting examples moved over, so let me get reup in.
- Mongo connection shutdown in main/start / stop
- Switch start to passing through stop function
Sometimes when you have to restart the server it’s a huge PITA because you can’t get the repl back up until you fix the errors. This causes your eval cycle time to jump to 10s of seconds.
Yup, had to do 7/8 edits to fix the problem, ran into a side case where generated cljs (and cljc) files were referencing a removed library (sablono). Why this was reported through the reup.clj path I have no idea. Tests maybe? nsfw.reup/setup?
- Highlighting broken on var page, check
+
- Check search results page
Something’s been bugging me about the ops/kit stuff. It’s really nice to be able to not think about the state transitions, to handle sync and async updates well, to handle context well. But something bugs me about the way you do updates:
(defn handle-update-example [state {:keys [text _id]}]
(let [ch (chan)]
(go
(put! ch (-> state
(update-example _id
merge
{:loading? true})))
(let [res (<! (req-update-example _id text))]
(put! ch
(fn [state]
(if (:success res)
(update-example
state
_id
merge
(:data res)
{:loading? false :editing? false})
(update-example
state
_id
merge
{:error (:message res)
:loading? false})))))
(close! ch))
ch))
Maybe I don’t know enough about core.async, but the puts seem redundant. Shouldn’t whatever the last statement evaluates to be put on the channel the go block returns? Do go blocks return channels, or did I make that up?
The larger issue is that my lack of understanding means I’m possibly missing or misusing some of the functionality that solves some of the problems I’ve been having composing async code.
Yup, go blocks return a channel with the last value calculated from the body:
`Returns a channel which will receive the result of the body when completed`
Anyways, what don’t I like about the block above?
- Have to remember to close the channel, if you forget it stays open forever.
- Feels ugly. Why? The meat of the post-backend-request code is the updating of state, but there’s a lot of ceremonoy there: Put value on `ch`, wrap in function, provide where to update the data.
- Mixing place-oriented and other stuff.
For the entire scope of the function we’re only updating the target example.
So I could create a function that closes over some of the repeated info:
(defn handle-update-example [state {:keys [text _id]}]
(let [ch (chan)
update-cur-example (fn [state & maps]
(update-example
state
_id
#(apply merge % maps)))]
(put! ch (uef state {:loading? true}))
(go
(let [res (<! (req-update-example _id text))]
(put! ch
(fn [state]
(if (:success res)
(update-cur-example state
(:data res)
{:loading? false :editing? false})
(update-cur-example state
{:error (:message res)
:loading? false})))))
(close! ch))
ch))
This doesn’t really feel like an improvement. Just moving the problem around. You do save a few lines, but is it more readable?
Next, have a `set-in-example` function that mirrors the clojure.core assoc / update pairs:
(defn handle-update-example [state {:keys [text _id]}]
(let [ch (chan)]
(put! ch (set-in-example state _id {:loading? true}))
(go
(let [res (<! (req-update-example _id text))]
(put! ch
(fn [state]
(if (:success res)
(set-in-example state _id
(:data res)
{:loading? false :editing? false})
(set-in-example state
{:error (:message res)
:loading? false})))))
(close! ch))
ch))
This does feel more readable, and the `set-in-example` function is probably useful in multiple contexts. `merge-with-example` is probably a better name. The tough thing with these functions is that they’re hard-coding the location of examples, but that shouldn’t change.
Meh.
2nd problem. I’ve got a single component – example editor –that’s used in two different contexts: when the user is creating examples and when they’re editing examples. The final action is to either create or update, but the component only has one signal (::save). I’m handling the different cases using top-level logic (create if no _id, update if _id), but this information is also available to the component. So these two cases are still hard-coded. Kind of?
Anyway, onto see alsos. I remember the UI I did last time was wonky, time to fix that.
Ok, so throttle / debounce on the component side or on the logic side? First hunch was on the logic side, but maybe this is an option that should be on the component.
On the SA UI, why is there an add button? The add buttons should be on the AC results (can’t add a SA to a var that doesn’t exist). Yay!
Rolling through GH issues and PRs. Been yak shaving for an hour trying to get an old enough version of mongo installed on my local. Also OSX now taking port 5k is causing issues, will switch to 4k, but can’t seem to find where this is set. ENV var PORT, but if not set should default to 8080 not 5k.
Well, that’s enought of that for now. Runnning into the
“` java.lang.IllegalArgumentException: No implementation of method: :urls of protocol: #’clojure.java.classpath/URLClasspath found for class: jdk.internal.loader.ClassLoaders$AppClassLoader “`
issue and trying to work through it. Time to get back to work, I’ll pick this up tomorrow.
Ok, finally got the prod-local server running. Tried a rollback to openjdk@11 to fix the protocol error above, but no joy. It has something to do with clojure.tool.namespace.repl’s use of classpath stuff, but I haven’t had a chance to dig into it. Full fix would be pulling all that stuff out probably.
Anyways got the server up and running, but encoutering js errors causing the page to not load completely. FastClick not defined and cannot read properties of undefined clojure.core. Not fun will have to dig in to get these figured out. Not great becuase any version bumps mean I’ll have to verify resource versions on the server which probably means a server rebuild.