Skip to content

Commit

Permalink
Making bigquery subpackage into a proper package.
Browse files Browse the repository at this point in the history
- Adding README, setup.py, MANIFEST.in, .coveragerc and tox.ini
- Adding google-cloud-bigquery as a dependency to the umbrella
  package
- Adding the bigquery subdirectory into the list of packages
  for verifying the docs
- Incorporating the bigquery subdirectory into the umbrella
  coverage report
- Adding the bigquery only tox tests to the Travis config
- Adding {toxinidir}/../core as a dependency for the bigquery
  tox config
  • Loading branch information
dhermes committed Sep 24, 2016
1 parent baca3e2 commit 6b7e52e
Show file tree
Hide file tree
Showing 9 changed files with 214 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,14 @@ install:
script:
- tox -e py27
- (cd core && tox -e py27)
- (cd bigquery && tox -e py27)
- tox -e py34
- (cd core && tox -e py34)
- (cd bigquery && tox -e py34)
- tox -e lint
- tox -e cover
- (cd core && tox -e cover)
- (cd bigquery && tox -e cover)
- tox -e system-tests
- tox -e system-tests3
- scripts/update_docs.sh
Expand Down
11 changes: 11 additions & 0 deletions bigquery/.coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[run]
branch = True

[report]
fail_under = 100
show_missing = True
exclude_lines =
# Re-enable the standard pragma
pragma: NO COVER
# Ignore debug-only repr
def __repr__
4 changes: 4 additions & 0 deletions bigquery/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
include README.rst
graft google
graft unit_tests
global-exclude *.pyc
89 changes: 89 additions & 0 deletions bigquery/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
Python Client for Google Cloud BigQuery
=======================================

Python idiomatic client for `Google Cloud BigQuery`_

.. _Google Cloud BigQuery: https://cloud.google.com/bigquery/what-is-bigquery

- `Homepage`_
- `API Documentation`_

.. _Homepage: https://googlecloudplatform.github.io/google-cloud-python/
.. _API Documentation: http://googlecloudplatform.github.io/google-cloud-python/

Quick Start
-----------

::

$ pip install --upgrade google-cloud-bigquery

Authentication
--------------

With ``google-cloud-python`` we try to make authentication as painless as
possible. Check out the `Authentication section`_ in our documentation to
learn more. You may also find the `authentication document`_ shared by all
the ``google-cloud-*`` libraries to be helpful.

.. _Authentication section: http://google-cloud-python.readthedocs.io/en/latest/google-cloud-auth.html
.. _authentication document: https://github.com/GoogleCloudPlatform/gcloud-common/tree/master/authentication

Using the API
-------------

Querying massive datasets can be time consuming and expensive without the
right hardware and infrastructure. Google `BigQuery`_ (`BigQuery API docs`_)
solves this problem by enabling super-fast, SQL-like queries against
append-only tables, using the processing power of Google's infrastructure.

.. _BigQuery: https://cloud.google.com/bigquery/what-is-bigquery
.. _BigQuery API docs: https://cloud.google.com/bigquery/docs/reference/v2/

Load data from CSV
~~~~~~~~~~~~~~~~~~

.. code:: python
import csv
from google.cloud import bigquery
from google.cloud.bigquery import SchemaField
client = bigquery.Client()
dataset = client.dataset('dataset_name')
dataset.create() # API request
SCHEMA = [
SchemaField('full_name', 'STRING', mode='required'),
SchemaField('age', 'INTEGER', mode='required'),
]
table = dataset.table('table_name', SCHEMA)
table.create()
with open('csv_file', 'rb') as readable:
table.upload_from_file(
readable, source_format='CSV', skip_leading_rows=1)
Perform a synchronous query
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python
# Perform a synchronous query.
QUERY = (
'SELECT name FROM [bigquery-public-data:usa_names.usa_1910_2013] '
'WHERE state = "TX"')
query = client.run_sync_query('%s LIMIT 100' % QUERY)
query.timeout_ms = TIMEOUT_MS
query.run()
for row in query.rows:
print row
See the ``google-cloud-python`` API `BigQuery documentation`_ to learn how
to connect to BigQuery using this Client Library.

.. _BigQuery documentation: https://googlecloudplatform.github.io/google-cloud-python/stable/bigquery-usage.html
68 changes: 68 additions & 0 deletions bigquery/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Copyright 2016 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os

from setuptools import find_packages
from setuptools import setup


PACKAGE_ROOT = os.path.abspath(os.path.dirname(__file__))

with open(os.path.join(PACKAGE_ROOT, 'README.rst')) as file_obj:
README = file_obj.read()

# NOTE: This is duplicated throughout and we should try to
# consolidate.
SETUP_BASE = {
'author': 'Google Cloud Platform',
'author_email': 'jjg+google-cloud-python@google.com',
'scripts': [],
'url': 'https://github.com/GoogleCloudPlatform/google-cloud-python',
'license': 'Apache 2.0',
'platforms': 'Posix; MacOS X; Windows',
'include_package_data': True,
'zip_safe': False,
'classifiers': [
'Development Status :: 4 - Beta',
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Topic :: Internet',
],
}


REQUIREMENTS = [
'google-cloud-core',
]

setup(
name='google-cloud-bigquery',
version='0.20.0dev',
description='Python Client for Google BigQuery',
long_description=README,
namespace_packages=[
'google',
'google.cloud',
],
packages=find_packages(),
install_requires=REQUIREMENTS,
**SETUP_BASE
)
30 changes: 30 additions & 0 deletions bigquery/tox.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[tox]
envlist =
py27,py34,py35,cover

[testing]
deps =
{toxinidir}/../core
pytest
covercmd =
py.test --quiet \
--cov=google.cloud.bigquery \
--cov=unit_tests \
--cov-config {toxinidir}/.coveragerc \
unit_tests

[testenv]
commands =
py.test --quiet {posargs} unit_tests
deps =
{[testing]deps}

[testenv:cover]
basepython =
python2.7
commands =
{[testing]covercmd}
deps =
{[testenv]deps}
coverage
pytest-cov
1 change: 1 addition & 0 deletions scripts/verify_included_modules.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
])
PACKAGES = (
'',
'bigquery',
'core',
)

Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@


REQUIREMENTS = [
'google-cloud-bigquery',
'google-cloud-core',
]

Expand Down
7 changes: 7 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ envlist =
[testing]
deps =
{toxinidir}/core
{toxinidir}/bigquery
pytest
covercmd =
py.test --quiet \
Expand All @@ -18,6 +19,12 @@ covercmd =
--cov-append \
--cov-config {toxinidir}/.coveragerc \
core/unit_tests
py.test --quiet \
--cov=google.cloud \
--cov=unit_tests \
--cov-append \
--cov-config {toxinidir}/.coveragerc \
bigquery/unit_tests
coverage report --show-missing --fail-under=100

[testenv]
Expand Down

0 comments on commit 6b7e52e

Please sign in to comment.