[[hadoop-config-and-shell]]
Configuration and Shell Usage
-----------------------------

The source of any Titan-Hadoop job is a `HadoopGraph`. `HadoopGraph` is simply a wrapper around a collection of Hadoop-specific configuration. Most importantly, it captures the location and type of the input graph and output graph. A `HadoopGraph` is typically created using one of the `HadoopFactory.open()` methods.

HadoopGraph Construction
~~~~~~~~~~~~~~~~~~~~~~~~

A Titan-Hadoop configuration file is used to construct a `HadoopGraph`. Assume a file named `conf/hadoop/titan-graphson.properties` as represented below.

[source,properties]
----
# input graph parameters
titan.hadoop.input.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONInputFormat
titan.hadoop.input.location=examples/graph-of-the-gods.json

# output data parameters
titan.hadoop.output.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONOutputFormat
titan.hadoop.sideeffect.format=org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
titan.hadoop.jobdir.location=jobs
titan.hadoop.jobdir.overwrite=true
----

With `HadoopFactory`, a configuration file is turned in a `HadoopGraph`. The `toString()` of the `HadoopGraph` denotes the input and output format of the graph. For instance, as seen below, a graph of type GraphSON is the input and a graph of type GraphSON is the output.

[source,gremlin]
gremlin> g = HadoopFactory.open('conf/hadoop/titan-graphson.properties')
==>hadoopgraph[graphsoninputformat->graphsonoutputformat]

HadoopGraph Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^

A `HadoopGraph` encapsulates an `org.apache.hadoop.conf.Configuration` instance.  This `Configuration` contains a mix of Hadoop and Titan configuration data.  This `Configuration` is instantiated with default resource loading enabled.  In other words, the it includes settings from `core-default.xml`, `mapred-default.xml`, `core-site.xml`, `mapred-site.xml`, etc., so long as those files are somewhere on the `CLASSPATH`.  It will also include any settings in the file provided to `HadoopFactory.open`.  Titan-Hadoop's configuration keys all start with "titan.hadoop" to mitigate the risk of collision with other keys sharing the `Configuration`.

The following snippet lists some typical `Configuration` keys, showing a mix of native Hadoop config keys and Titan-Hadoop config keys. 

[source,gremlin]
gremlin> g = HadoopFactory.open('conf/hadoop/titan-graphson.properties')
==>titangraph[hadoop:graphsoninputformat->graphsonoutputformat]
gremlin> g.getConf()
==>dfs.datanode.data.dir=file://${hadoop.tmp.dir}/dfs/data
==>dfs.namenode.checkpoint.txns=1000000
==>s3.replication=3
==>titan.hadoop.input.location=examples/graph-of-the-gods.json
...

It's possible to retrieve a read-only view of a subset of the configuration by calling `HadoopGraph.getConf(String prefix)`:

[source,gremlin]
gremlin> g.getConf('mapred')
==>mapred.disk.healthChecker.interval=60000
==>mapred.task.tracker.http.address=0.0.0.0:50060
==>mapred.userlog.retain.hours=24
==>mapred.local.dir.minspacestart=0
==>mapred.cluster.reduce.memory.mb=-1
==>mapred.reduce.parallel.copies=5
...

[source,gremlin]
gremlin> g.getConf('titan.hadoop')
==>titan.hadoop.input.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONInputFormat
==>titan.hadoop.input.location=examples/graph-of-the-gods.json
==>titan.hadoop.output.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONOutputFormat
==>titan.hadoop.sideeffect.format=org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
gremlin>

See <<hadoop-config-ref>> for a list of Titan-specific configuration keys (i.e. keys that start with "titan.hadoop").

Titan-Hadoop Properties
^^^^^^^^^^^^^^^^^^^^^^^
 
`HadoopGraph` provides getters/setters for mutating some commonly used properties. 

[source,gremlin]
gremlin> g = HadoopFactory.open('conf/hadoop/titan-graphson.properties')
==>titangraph[hadoop:graphsoninputformat->graphsonoutputformat]
gremlin> g.getGraphOutputFormat()
==>class com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONOutputFormat
gremlin> g.setGraphOutputFormat(NoOpOutputFormat.class)
==>null
gremlin> g
==>titangraph[hadoop:graphsoninputformat->noopoutputformat]
gremlin> g.getGraphOutputFormat()
==>class com.thinkaurelius.titan.hadoop.formats.noop.NoOpOutputFormat

Chaining Graphs
^^^^^^^^^^^^^^^

`HadoopGraph` provides a `getNextGraph()` method. This generates a new `HadoopGraph` that is the "inverse" of the current with the input formats and output locations reconfigured to support easy graph chaining.

[source,gremlin]
gremlin> g = HadoopFactory.open('conf/hadoop/titan-graphson.properties')
==>titangraph[hadoop:graphsoninputformat->graphsonoutputformat]
gremlin> h = g.getNextGraph()
==>titangraph[hadoop:graphsoninputformat->graphsonoutputformat]
gremlin> h.getConf('titan.hadoop')
==>titan.hadoop.input.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONInputFormat
==>titan.hadoop.jobdir.location=jobs_
==>titan.hadoop.input.location=jobs
==>titan.hadoop.output.format=com.thinkaurelius.titan.hadoop.formats.graphson.GraphSONOutputFormat
==>titan.hadoop.sideeffect.format=org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
