Skip to content

Commit

Permalink
readme clarification
Browse files Browse the repository at this point in the history
  • Loading branch information
wresch committed May 12, 2018
1 parent 01a1a5f commit 42baedc
Showing 1 changed file with 4 additions and 9 deletions.
13 changes: 4 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,13 @@ Cluster id Slurm jobid state
```

As you can see there is a ssh command that will set up a tunnel
between your local machine, and the spark cluster. Execute that
command in a terminal on your desktop if you have an os x or
linux machine.
If you have a windows box, you can eithe use putty to set up the
port forwarding or you can use NX to connect to helix and then use that
command in your nx session.
between your local machine, and the spark cluster if you want
to see the spark master's web ui.

Next, on your local machine (or your helix nx session), open up a
browser and point it to 'localhost:55555' to see the spark web ui.

Here is how you would connect to the cluster with pyspark using one executor
per node:

Connect to the cluster with pyspark using one executor per node.
```
$ pyspark --master spark://cn0443:7077 --executor-memory=40g
Welcome to
Expand Down

0 comments on commit 42baedc

Please sign in to comment.