GCJ Command Line v1.0-beta4, released 2011-06-03

[Feature]: Source files specified in the configuration files and using the -a
           option in gcj_submit_solution.py can now be glob patterns.

[Feature]: In the tools/ directory there is an update_hashbangs.py script to
           change the hashbang line of a bulk of python files using a single
           command.

[Fix]: --base_dir option was not working in gcj_submit_solution.py.

[Fix]: Default value for --base_dir was not following symlinks properly.

[Fix]: User status script was crashing when the user is not logged in or did
       not participate in the contest.

[Fix]: Configuration file reading was crashing when spaces were inserted before
       the opening brace. A fix was made to clean the configuration file before
       attempting to parse it.

Author: jorges

CHANGES

@@ -1,3 +1,24 @@
+-------------------------------------------------------------------------------
+GCJ Command Line v1.0-beta4, released 2011-06-03
+
+[Feature]: Source files specified in the configuration files and using the -a
+           option in gcj_submit_solution.py can now be glob patterns.
+
+[Feature]: In the tools/ directory there is an update_hashbangs.py script to
+           change the hashbang line of a bulk of python files using a single
+           command.
+
+[Fix]: --base_dir option was not working in gcj_submit_solution.py.
+
+[Fix]: Default value for --base_dir was not following symlinks properly.
+
+[Fix]: User status script was crashing when the user is not logged in or did
+       not participate in the contest.
+
+[Fix]: Configuration file reading was crashing when spaces were inserted before
+       the opening brace. A fix was made to clean the configuration file before
+       attempting to parse it.
+
 -------------------------------------------------------------------------------
 GCJ Command Line v1.0-beta3, released 2011-05-12
 

README

@@ -1,4 +1,4 @@
-GCJ Commandline submit tool, v1.0-beta3
+GCJ Commandline submit tool, v1.0-beta4
 Copyright 2010 Google Inc. All Rights Reserved.
 
 Download and submit tool for the Google Code Jam
@@ -127,8 +127,8 @@ which will download the input file and store it in source/A-small-0.in. Now
 it is time to run your solution and generate the output file.
 
 NOTE: This script downloads an input file for the specified problem and starts
-its timer without asking a confirmation. You should double-check the parameters
-before running it or you might start a timer for another problem.
+its timer without asking for confirmation. You should double-check the
+parameters before running it or you might start a timer for another problem.
 
 NOTE: Downloading an input file twice in the same attempt will just redownload
 the same input file if its timer has *NOT* expired. Otherwise, it will download
@@ -155,9 +155,9 @@ to submit two extra files called library.cpp and template.txt you can execute:
 $ ./gcj_submit_solution.py -a library.cpp -a template.txt A small 0
 
 which will submit the file source/A-small-0.out as the answer and
-source/A-small-0.cpp and library.cpp as the source files. If you add a
-directory using the -a option, the program will compress it into a zip file
-before submitting it.
+source/A-small-0.cpp, library.cpp and template.txt as the source files. If you
+add a directory using the -a option, the program will compress it into a zip
+file before submitting it.
 
 After submitting the file, the Code Jam server will answer with a message
 string. The most important values are:
@@ -176,7 +176,7 @@ string. The most important values are:
 
 During the contest you can check your status by using the gcj_get_status.py
 script, which will show you your current rank, number of points, time at which
-input has been solved, number of wrong attempts per input and remaining time for
+each input was solved, number of wrong attempts per input and remaining time for
 current attempts.
 
 To run this script you should execute:

__init__.py

@@ -1,4 +1,4 @@
-#!/usr/bin/python2
+#!/usr/bin/env python2
 # -*- coding: utf-8 -*-
 #
 # Copyright 2011 Google Inc.

gcj_clear_contest.py

@@ -47,7 +47,7 @@ def main():
     parser.add_option('--base_dir', action='store', dest='base_dir',
                       help=('Base directory used to parametrize configuration '
                             'file paths'))
-    parser.set_defaults(base_dir=os.path.dirname(__file__))
+    parser.set_defaults(base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

gcj_download_input.py

@@ -67,7 +67,7 @@ def main():
                       help=('Base directory used to parametrize configuration '
                             'file paths'))
     parser.set_defaults(renew_cookie=False, force=False,
-                        base_dir=os.path.dirname(__file__))
+                        base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

gcj_get_status.py

@@ -94,7 +94,8 @@ def main():
     parser.add_option('--base_dir', action='store', dest='base_dir',
                       help=('Base directory used to parametrize configuration '
                             'file paths'))
-    parser.set_defaults(renew_cookie=False, base_dir=os.path.dirname(__file__))
+    parser.set_defaults(renew_cookie=False,
+                        base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

gcj_init_contest.py

@@ -46,7 +46,7 @@ def main():
     parser.add_option('--base_dir', action='store', dest='base_dir',
                       help=('Base directory used to parametrize configuration '
                             'file paths'))
-    parser.set_defaults(base_dir=os.path.dirname(__file__))
+    parser.set_defaults(base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

gcj_renew_login.py

@@ -46,7 +46,7 @@ def main():
     parser.add_option('--base_dir', action='store', dest='base_dir',
                       help=('Base directory used to parametrize configuration '
                             'file paths'))
-    parser.set_defaults(base_dir=os.path.dirname(__file__))
+    parser.set_defaults(base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

gcj_submit_solution.py

@@ -93,7 +93,7 @@ def main():
     parser.set_defaults(renew_login=False, force=False, gzip_content=True,
                         zip_sources=False, ignore_zip=False,
                         ignore_def_source=False,
-                        base_dir=os.path.dirname(__file__))
+                        base_dir=os.path.dirname(os.path.realpath(__file__)))
     options, args = parser.parse_args()
 
     # Store the script location in a runtime constant, so it can be used by

lib/output_submitter.py

@@ -19,6 +19,7 @@
 
 
 
+import glob
 import httplib
 import json
 import os
@@ -51,14 +52,16 @@ def __init__(self, host, cookie, middleware_token, contest_id, problem_id):
     self.contest_id = contest_id
     self.problem_id = problem_id
 
-  def _PrepareSourceFiles(self, source_names):
+  def _PrepareSourceFiles(self, source_patterns):
     """Zip all source directories into files and return them.
 
     The returned list will contain (filename, file_data) tuples, where file_data
     will be None for files that have not been read yet.
 
     Args:
-      source_names: List with all the files to include with the solution.
+      source_patterns: List with all the file patterns to include with the
+        solution. These file patterns will be expanded using python's glob
+        module.
 
     Returns:
       A (source_files, ignored_zips) tuple, where source_files is a list of
@@ -70,23 +73,25 @@ def _PrepareSourceFiles(self, source_names):
     source_files = []
     ignored_zips = set()
 
-    # Process each source specified by the user.
-    for source in source_names:
-      # Check if the source is a directory or a file.
-      if os.path.isdir(source):
-        # Create a zip file in memory for the directory, add it to the source
-        # files and update the ignored_zips set.
-        sys.stdout.write('Compressing directory "{0}"...\n'.format(source))
-        zipped_contents, newly_ignored_zips = (
-            zip_utils.MakeZipFileInMemory([source], ignore_exts=['.zip']))
-        ignored_zips.update(newly_ignored_zips)
-        flat_source_name = source.replace('\\', '_').replace('/', '_')
-        zip_filename = '{1}_{0}.zip'.format(random.randrange(0, 2**31 - 1),
-                                            flat_source_name)
-        source_files.append((zip_filename, zipped_contents))
-      else:
-        # Add files directly to the prepared sources.
-        source_files.append((source, None))
+    # Process each source pattern specified by the user, expanding them using
+    # the python's glob module.
+    for source_pattern in source_patterns:
+      for source in glob.iglob(source_pattern):
+        # Check if the source is a directory or a file.
+        if os.path.isdir(source):
+          # Create a zip file in memory for the directory, add it to the source
+          # files and update the ignored_zips set.
+          sys.stdout.write('Compressing directory "{0}"...\n'.format(source))
+          zipped_contents, newly_ignored_zips = (
+              zip_utils.MakeZipFileInMemory([source], ignore_exts=['.zip']))
+          ignored_zips.update(newly_ignored_zips)
+          flat_source_name = source.replace('\\', '_').replace('/', '_')
+          zip_filename = '{1}_{0}.zip'.format(random.randrange(0, 2**31 - 1),
+                                              flat_source_name)
+          source_files.append((zip_filename, zipped_contents))
+        else:
+          # Add files directly to the prepared sources.
+          source_files.append((source, None))
 
     # Return all generated sets.
     return source_files, ignored_zips
@@ -123,15 +128,16 @@ def _ParseResult(self, response_data, input_public):
                               'cannot submit solution. Check that the host, '
                               'user and contest id are valid: {0}.\n'.format(e))
 
-  def Submit(self, input_id, output_name, source_names, input_public,
+  def Submit(self, input_id, output_name, source_patterns, input_public,
              gzip_body=True, zip_sources=False, add_ignored_zips=False):
     """Submit the specified output and sources file to the problem.
 
     Args:
       input_id: Identifier of the output to submit ('0' for the small output,
         '1' for the large output).
       output_name: Name of the file with the output data.
-      source_names: Names of the source files to be included with the output.
+      source_patterns: Name patterns of the source files to be included with the
+        output. These patterns will be expanded using Python's glob module.
       input_public: Boolean indicating whether the answer is public or not.
       gzip_body: Boolean indicating whether the body has to be gzipped or not.
       zip_sources: Boolean indicating whether all sources should be put inside a
@@ -150,7 +156,7 @@ def Submit(self, input_id, output_name, source_names, input_public,
     # Prepare the source files (zipping all directories). After this,
     # source_files will only contain text files and zip files specified directly
     # or by compressing a directory.
-    source_files, ignored_zips = self._PrepareSourceFiles(set(source_names))
+    source_files, ignored_zips = self._PrepareSourceFiles(set(source_patterns))
 
     # Check if the user requested to zip source files.
     if zip_sources:

tools/__init__.py

@@ -0,0 +1,18 @@
+#!/usr/bin/env python2
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+

tools/update_hashbangs.py

@@ -0,0 +1,130 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright 2011 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""This file implements the a tool to change the hashbang line of a bulk of
+python files."""
+
+
+
+import optparse
+import os
+import sys
+
+
+def ChangeFileHashbang(filename, new_hashbang_line):
+  """Change the specified file hashbang line.
+
+  This function assumes that the specified filename is a python file.
+
+  Args:
+    filename: String with the name of the file to modify.
+    new_hashbang_line: String with the new hashbang line to put in the file.
+  """
+  # Open the file and read all lines from it.
+  with open(filename, 'rt') as file_obj:
+    file_lines = file_obj.readlines()
+
+  # Substitute the first line if it's an existing hashbang. Otherwise add the
+  # new hashbang at the top of the file.
+  complete_hashbang_line = '#!{0}\n'.format(new_hashbang_line.strip())
+  if len(file_lines) > 0 and file_lines[0].startswith('#!'):
+    file_lines[0] = complete_hashbang_line
+  else:
+    file_lines[0:1] = [complete_hashbang_line]
+
+  # Write the processed lines to the file.
+  with open(filename, 'wt') as file_obj:
+    file_obj.write(''.join(file_lines))
+
+
+def ProcessFile(target, new_hashbang_line):
+  """Change the hashbang line of the specified target if it's a python file.
+
+  Args:
+    target: String with the file to modify.
+    new_hashbang_line: String with the new hashbang line to put in the target.
+  """
+  # Only process a target if it's a python file.
+  if os.path.isfile(target):
+    extension = os.path.splitext(target)[1]
+    if extension == '.py':
+      print 'Adding hashbang to {0}.'.format(target)
+      ChangeFileHashbang(target, new_hashbang_line)
+    else:
+      print 'Ignoring {0}, it\'s not a python file.'.format(target)
+  elif not os.path.isdir(target):
+    print 'Ignoring {0}, it\'s not a file.'.format(target)
+
+
+def ProcessTarget(target, new_hashbang_line, recursive):
+  """Process a target, which might be a directory or a regular file.
+
+  Args:
+    target: Name of the directory or file to process.
+    new_hashbang_line: String with the new hashbang line to put in the target.
+    recursive: Flag indicating whether directories should be processed
+      recursively or not.
+  """
+  # Process directories recursively or not depending on the recursive flag, if
+  # the recursive flag is False then only directly inside the target are
+  # processed. Target files are processed directly.
+  target_files = []
+  if os.path.isdir(target):
+    if recursive:
+      for dirpath, dirnames, filenames in os.walk(target):
+        target_files.extend(os.path.join(dirpath, filename)
+                            for filename in filenames)
+    else:
+      target_files.extend(os.path.join(target, filename)
+                          for filename in os.listdir(target))
+  else:
+    target_files.append(target)
+
+  # Process each target file independently.
+  for filename in target_files:
+    ProcessFile(filename, new_hashbang_line)
+
+
+def main():
+  """Main function for the update hashbang script.
+
+  This script receives two positional arguments or: the new hashbang to assign
+  to all python files and a list of one or more directories with all the files
+  to modify.
+  """
+  # Create an option parser and use it to parse the supplied arguments.
+  program_version = 'GCJ hashbang tool'
+  parser = optparse.OptionParser(usage='%prog [options]',
+                                 version=program_version)
+  parser.add_option('-r', action='store_true', dest='recursive',
+                    help=('Process all directories recursively'))
+  parser.set_defaults(recursive=False)
+  options, args = parser.parse_args()
+
+  # Check that the number of arguments is valid.
+  if len(args) < 2:
+    sys.stderr.write('need at least two positional arguments\n')
+    sys.exit(1)
+
+  # Modify all targets applying the new hashbang line.
+  new_hashbang_line = args[0]
+  for target in args[1:]:
+    ProcessTarget(target, new_hashbang_line, options.recursive)
+
+
+if __name__ == '__main__':
+  main()