block design

doc/design/block.md

@@ -0,0 +1,215 @@
+# Design Doc: Use Block in RNNOp, While Op, IfElseOp
+
+In C++ and Java programming language, a block is a lexical structure of source code which is grouped as one line of code.
+
+RNNOp looks like the loop structure in programming languages.
+And similarly, WhileOp and IfElseOp are like loop and conditions respectively.
+So we want to verify if we should have a class Block in PaddlePaddle that works like a pair of curly braces in the loop and condition structures of programming languages.
+
+Blocks do not only group source code, but also narrow the lexical scope of variables so that they do not conflict with variables having the same name used elsewhere in a program.
+
+In Paddle, we need a similar concept called Block to support following scenes:
+
+- define a PaddlePaddle program by writing blocks of codes, which includes the definitions of variables and operators.
+  - `RNNOp`, `SwitchOp`, `WhileOp` and `IfElseOp`, etc, need Block to help to define sub-block. 
+- help to execute multiple operators, blocks should group operators and runs like a single operator.
+
+## How to use Block
+In `RNNOp`, `SwitchOp`, `WhileOp` and `IfElseOp`, a with-statement should be used to help to define a sub-block.
+
+Let's start from how a RNNOp is described using Block:
+
+```python
+v = some_op()
+m_boot = some_op()
+
+W = pd.Variable(shape=[20, 20])
+U = pd.Varable(shape=[20, 20])
+
+rnn = create_rnn()
+with rnn.stepnet() as net:
+  # declare the input variables that need to be segmented into steps
+  x = net.set_inputs(v)
+  # declare rnn's memory (state)
+  h = net.add_memory(init=m_boot)
+
+  fc_out = pd.matmul(W, x)
+  hidden_out = pd.matmul(U, h.pre(n=1))
+  sum = pd.add_two(fc_out, hidden_out)
+  act = pd.sigmoid(sum)
+  h.update(act) # update memory
+
+  # declare outputs that needs to be merged across all the steps
+  net.set_outputs(act, hidden_out)
+
+acts, hs = rnn()
+```
+
+The with-statement above describes a `RNNOp`'s stepnet as a block, this description will be transformed to a protobuf message as follows 
+
+```
+BlockDesc RNNOp_stepnet {
+  vars = {
+    x {...}
+    h {...}
+    fc_out {...}
+    hidden_out {...}
+    sum {...}
+    act {...}
+  }
+
+  ops = {
+    matmul,
+    add_two,
+    sigmoid
+  }
+};
+
+RNNOpDesc rnn {
+  inputs = {x};
+  outputs = {act, hidden_out};
+  attrs { memories={h} };
+  stepnet {RNNOp_stepnet};
+};
+```
+
+and pass it to a C++ Block, the C++ Block will create the Variables and Operators.
+
+
+## Block Implementation
+
+During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
+
+VarDesc in a block should have its name scope to avoid local variables affect parent block's name scope. 
+Child block's name scopes should inherit the parent's so that OpDesc in child block can reference a VarDesc that stored in parent block. For example
+
+```python
+a = pd.Varaible(shape=[20, 20])
+b = pd.fc(a, params=["fc.w", "fc.b"])
+
+rnn = pd.create_rnn()
+with rnn.stepnet() as net:
+    x = net.set_inputs(a)
+    # reuse fc's parameter
+    fc_without_b = pd.get_variable("fc.w")
+    net.set_outputs(fc_without_b)
+
+out = rnn()
+```
+the method `pd.get_variable` can help retrieve a Variable by a name, a Variable may store in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
+
+In compiler design, the symbol table is an data structure created and maintained by compilers in order to store information about the occurrence of various entities such as variable names, function names, classes, etc.
+
+To store the definition of Variables and Operators, a C++ class `SymbolTable` is introduced as a similar concept with compiler's symbol table.
+
+`SymbolTable` will has following functions:
+
+- store the definitions (some names and attributes) of variables and operators,
+- to verify if a variable name has been declared,
+- to make it possible to implement type checking (offer Protobuf message pointers to `InferShape` handlers).
+
+
+```c++
+// Information in SymbolTable is enough to trace the dependency graph. So maybe
+// the Eval() interface takes a SymbolTable is enough.
+class SymbolTable {
+ public:
+  SymbolTable(SymbolTable* parent) : parent_(parent) {}
+
+  OpDesc* NewOp(const string& name="");
+
+  // TODO determine whether name is generated by python or C++
+  // currently assume that a unique name will be generated by C++ if the
+  // argument name left default.
+  VarDesc* NewVar(const string& name="");
+
+  // find a VarDesc by name, if recursive true, find parent's SymbolTable
+  // recursively.
+  // this interface is introduced to support InferShape, find protobuf messages
+  // of variables and operators, pass pointers into InferShape.
+  // operator
+  //
+  // NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
+  // be proposed and embedded into pybind to enable python operate on C++ pointers.
+  VarDesc* FindVar(const string& name, bool recursive=true);
+
+  OpDesc* FindOp(const string& name);
+
+  BlockDesc Compile() const;
+
+ private:
+  SymbolTable* parent_;
+
+  map<string, OpDesc> ops_;
+  map<string, VarDesc> vars_;
+};
+```
+
+After all the description of variables and operators is added into SymbolTable,
+the block has enough information to run.
+
+The `Block` class takes a `BlockDesc` as input, and provide `Run` and `InferShape` functions.
+
+
+```c++
+namespace {
+
+class Block : OperatorBase {
+public:
+  Block(const BlockDesc& desc) desc_(desc) {}
+
+  void InferShape(const framework::Scope& scope) const override {
+    if (!symbols_ready_) {
+      CreateVariables(scope);
+      CreateOperators();
+    }
+    // should run InferShape first.
+    for (auto& op : runtime_table_.ops()) {
+      op->InferShape(scope);
+    }
+  }
+
+  void Run(const framework::Scope& scope,
+           const platform::DeviceContext& dev_ctx) const override {
+    PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
+    for (auto& op : runtime_table_.ops()) {
+      op->Run(scope, dev_ctx);
+    }
+  }
+
+  void CreateVariables(const framework::Scope& scope);
+  void CreateOperators();
+
+  // some other necessary interfaces of NetOp are list below
+  // ...
+
+private:
+  BlockDesc desc_;
+  bool symbols_ready_{false};
+};
+```
+
+## Run and Eval targets
+Block inherits from OperatorBase, which has a Run method. 
+Block's Run method will run its operators sequentially.
+
+There is another important interface called `Eval`, which take some arguments called targets, and generate a minimal graph which takes targets as the end points and creates a new Block, 
+after `Run`, `Eval` will get the latest value and return the targets.
+
+The definition of Eval is as follows:
+
+```c++
+// clean a block description by targets using the corresponding dependency graph.
+// return a new BlockDesc with minial number of operators.
+// NOTE not return a Block but the block's description so that this can be distributed
+// to a cluster.
+BlockDesc Prune(const BlockDesc& desc, vector<string> targets);
+
+void Block::Eval(const vector<string>& targets,
+                 const framework::Scope& scope,
+                 const platform::DeviceContext& dev_ctx) {
+  BlockDesc min_desc = Prune(desc_, targets);
+  Block min_block(min_desc);
+  min_block.Run(scope, dev_ctx);
+}
+```