# Ed3

Shokunin - 職人
The digital workshop.

A set of tools to enable a variety of projects. These tools can be organized, modified, and extended by the craftsperson to enable new projects and methods.
A maleable space that can be optimized by the worker to match the tasks at hand. Even those tasks not imagined by the designer.

For a beginning, bring together the simplest, time tested tools to accomplish each task and provide the interface glue that places them most ready to hand in an ergonomical manner, while allowing the worker to customize this interface.

Focus on really good UX for a text editor then terminal and cells. Add hooks to build tooling around that UX.

A common language (Python) for extending and, as much as possible, configuring the workshop.

##Core
    1. Good UX for opening files/tools (text editor, cells, terminal, etc) and navigating open windows.
    2. Ability to run python scripts in the main interpreter easily, or run commands as external processes.
    3. Easy to add above scripts/commands to the UI.


##Window Management
Do I want to reimplement i3 in the main widget area OR rely on DockWidgets? Note that either way I need a good tabbed interface for having many open files. (ie. better than i3). I've already done both before...
    
##Finding Files (opening)
fzf style vs. QFileDialog.

##Running External Commands
This is fairly straight forward for line oriented output. I looked briefly into supporting curses immediate mode (or whatever it's called) and that looked like a non-starter without a lot of work or a VTE library.
For now I can run stuff in QProcess and pipe output to a QTextEdit.

##Running python scripts on the interpreter
I really like being able to write a small generator and run it with output sent to the QTextEdit. I need to look into how this
might work for editor and cells tools.

##Adding scripts and commands to the UI
I had a good setup for predefined Actions but adding user scripts/commands hasn't coalesced into something good yet. Additionally I'm struggling with the tension between Shortcuts, Menu items and command line invocations.


##Tasks to address in version 1.0

    - Writing code.
    - Compiling code.
    - Running a compiled program.
    - Writing prose.
    - Filing documents.
    - Searching/Retrieving files.
    - Running external tools (via shortcut, timer event, or on file change).


##TODO

        # TODO: style titlebar on windows?
        # TODO: revert file
        # TODO: toggle more prevalent keyboard hints (overlay?)
        # TODO: About page with attributions (icon design, Qt, etc)
        # TODO: Second window
        # TODO: virtual desktops
        # TODO: when closing, list all files w/ changes
        # TODO: when opening a project change CWD
        # TODO: bookmarks dock
        # TODO: format paragraph
        # TODO: spell check
        # TODO: pylint/flake8
        # TODO: Jedi integration: syntax check, autocomplete, goto def, refactoring, search
        # TODO: init VCS: dock with diff, commit, push
[ ] look into VCS ala Magit
        # TODO: CI: make.py, makefile, tests, dock widget with targets
        # TODO: add cells
        # TODO: factor out keybinding to config file
        # TODO: rename
        # TODO: interpreter
        # TODO: Kanban todo tracking
        # TODO: built in distributed bug tracking
        # TODO: review hyper-modern python for tool ideas
        # TODO: add wiki features (ie, zim & vimwiki)
        # TODO: color chooser widget

[/] Style scrollbars
[/] prompt string to terminal
[/] cancel open -> NOOP
[/] cd command
[/] Keys to resize dock
[/] Style margin
[/] why doesn't terminal remember it's location?
    Because I was trying to add it to MainGui using ctrl.gui, which is set to None during MainGui.__init__
    Had to pass in the MainGui instance explicitly.

[ ] get a good redistributable nerd fonts package (with symbols, etc)

PMPM (Poor Man's Project Management) features
[ ] macro to add a task
[ ] macro to replace open task box with complete or discarded
■ □ ☑ ☒ ☐
[ ] macro to open log with new entry

[ ] switch from tuples/DataClasses to attrs based classes

[ ] keys to move docks to different areas
[ ] keys to move focus spatially (ala i3)?
[ ] tab completion to terminal

[ ] back to last edited buffer (recall)

[ ] check for unknown symbols
[ ] check for unused imports

[ ] Vim mode
[ ] Emacs mode

[ ] PgUp/PgDown to PyHelper
[ ] Search to PyHelper

[ ] use blackd
    blackd parses black.py 100 times in 1.67s versus 14.73s for black.

[ ] develop UI for making threads more visible

### Linting features
[ ] Unused imports
[ ] undefined symbols
[ ] subclasses that don't call super().__init__()

Are these the same bug?
[/] focus border disappears sometimes, changing window size brings it back
[/] cursor disappears sometimes or doesn't blink
Seems to happen whenever a new file is loaded (even if a new Editor isn't created). I think it has to do with
QFileDialog not returning focus properly. But no combination of clearFocus, setFocus, or activateWindow (even with a 500ms timer) at the end of gui.open seems to fix it. Alt-tab away and back restores focus. The editor still has keyboard focus but so kind of graphical focus (border and blinking cursor) is missing.
Changing the parent of QFileDialog to none (it was already MainGui) had no affect. Using the native widget did fix it.
After a lot of work to subclass QFileDialog (it's API is weird) I found that the QCompleter pop-up was staying open and I could manually hide it by overriding the done method.


[/] Trigger PyHelp with keyword under cursor
[ ] use WaitCursor on load/save

[ ] add a filemanager dock
[ ] jump file navigation
[ ] fuzzy search (files and code)
    Add a fuzzy search open file command to OpenFilesDock

[ ] code navigation
[ ] code completion
[ ] look into Tree-sitter parsers
[ ] shortcut config file

[/] fix colors of table view headers (esp. in OpenFileDialog)
[/] unify application styling via css and TextEdit charFormats

[/] menu options for loading config files
[/] move pygments into a subrepo

[ ] have Open Files hide the project path
[ ] collapse user home directory in all displayed paths

[ ] show shortcuts in dock titles
[ ] autosave
[ ] update project view when files change
Should show M when a file changes
Also monitor for commit
Show current tag

[ ] notify user when open file has changed outside of editor
QFileSystemWatcher: connect fileChanged to editor

[ ] Command menu gen from config file
[ ] Running subprocess list
[ ] Command line searches actions

[ ] command to collate TODOs

## META-tool
Runs a script, with the currently open tool's contents as input and then displays the results in a new tool.


## Movement toward a generic Python execution engine
ala Emacs. I've never been an Emacs user but I've read that it is really a lisp programming 
environment/VM, in which the default tool is a pretty decent text editor. I'm thinking of
of something like that for the workshop.
Needs:
1. Integrated python distribution as part of the installation.
2. Good subprocess monitoring and isolation from the main process.
3. IPC mechanisms that scale with complexity. Streams might be good enough in a lot of cases. 
4. Easy to use json-rpc for more complex cases.
5. UI hooks for communicating to subprocesses.

One place this could be really useful is on systems that don't have, or easily support existing development environments. e.x. Arm based devices, phones, tablets, chromebooks. These are often served by web based environments but what if offline first was preferred? An integrated execution and development environment might have applications. Further, Juypter proved that integrating a python distribution with a development environment had utility to niche (or domain oriented) programmers.

## On *Actions*
Actions are things that the workshop can do. They can be triggered by the user in different ways, by selecting from the menu, hitting a shortcut key (or combo) and by typing a command into the command bar. A user should have the ability to configure the shortcuts for an action. There can also be user defined actions.

Actions are identified by an Enum, this allows static analysis. Turns out an Enum will happily let you redefine members at run time. I don't like that and classes don't appear to have a __setattr__ method that I could overwrite. I might be rolling my own class.

Actions are verbs. I want the user to be able to interact in noun/verb order. Specifically, the thing to operate on should be the focus first, then the operation or action to take. Examples of nouns are, the cursor with focus, the text selected, a filename in a listing, the current tool in focus, etc. Respective actions (verbs) might be move left, copy to clipboard, open in appropriate tool, move to left hand dock. An action might have optional parameters. I also want a common method to repeat an action N times. This gets to the VIM concept of a rich and composable command language.

Detailed examples:
    CursorInFocus Move Right 1 Character
    CursorInToolX Move [AndSelect] [Right|Left|Up|Down] N [Char|Word|Cell|Page|Block]
    ToolInFocus Move RightDock
    Focus Move RightDock


Nouns and verbs should be available from a command line interface. When actions are triggered via a menu or shortcut the corresponding command line arguments should be displayed.

There are three parts here:
    1. The command language.
    2. Menu/shortcut actions and how they map to the command language.
    3. Implementations of specific actions as they relate to specific nouns.

Where/how are each of these things defined and how are they bridged together?




## Meta programming hook (macros)
Pass current file/selection to script. Copy output to current text buffer.
Allow string, AST, or data model manipulation. Example: evaluate currently selected dictionary and extract keys as list, converting from strings to identifiers.


## Thoughts on QSS
Stylesheets are lovely because they offer tremendous flexibility with minimal code. The problem is that they are incredibly obtuse. Order matters in unexpected ways, selectors are bizarre, and syntax errors fail silently. Maybe I can arrive at some kind of middle ground, prehaps where styled objects are all named and so are explicitly selected (so that cascading and precendence don't cause unexpected weirdness), maybe run the stylesheet through some sort of validator... 

I'm hesitant to give up the power offered by incorporating stylesheets directly. But it really needs some structure.

How can I apply stylesheets to TextCharFormats? Since QTextCharFormat is not a widget, I don't think I can, not even with qproperty.

A compromise is to develop yaml + template QSS styling, but allow literal QSS to override.
Templating options:
- f-strings cannot handle the QSS blocks "{}" without esacping.
- string.Template uses $VAR but does not allow expressions.
- Jinja requires learning another DSL but has no problem with QSS blocks.

I decided to try leaning harder on python for the settings. I wound up using string.Template (although old school %-formatting would have worked too). The QSS is defined in settings.py and each style is a function that sets a number of variables used in the template substitution.

##Singletons vs. Controller pattern
Originally I had a controller object (Main) that maintained references to all the global state classes. ex. Dispatcher, Settings, etc. The controller was passed down to all child objects so they could refer to other things. This pattern creates strong (and often unnesscesary coupling).

Once I got comfortable with my Singleton implementation I realized I could move a lot of things that were previously referenced in the controller to Singletons. Then only import what was needed into each child class. I like how this decouples children from the controller and (by extension) to each other.

Order of import must be carefully maintained. Some things need to reference each other and so can't use the import mechanism. All my singletons have a .setup() method that I run once in main. And optionally a .connect() method for making these connections explicit. 

The hope is that this decoupling helps with testing and developing.





#BUGS
##black import error
BEWARE: import order matters
  > python3
  >>> import PySide2
  >>> import black
   File "src/black/trans.py", line 156, in <module>
   TypeError: mypyc classes can't have a metaclass
found an instance with PySide2 5.15.2.1, black 22.3.0, Python 3.9.2 Win64
where importing PySide2 before black caused black to fail on import
same on Linux with PySide 5.15.2, black 22.3.0 and Python 3.9.2 
also true w/ black==22.1.0
not an issue with black==21.9b0, however pyinstaller built exe fails to load
with an unhandled exception in black's import chain
pip install -U black==22.3.0 --no-binary black does not exhibit above import bug
but pyinstaller build still fails on windows.
Adding --collect-all blib2to3 to my pyinstaller build seems to help although
it fails to find the metadata for blib2to3


#FAQ


##License
