Utility Functions

Pyshortcuts provides a number of utility functions, especially for working with text files. These may seem like an assorted mix of functions. The author found many of these useful in multiple projects, and rather than having different versions in different packages, used this package to host those. The utilities here are small (adding no extra dependencies), but useful for many projects.

isotime(): get time is ISO format

pyshortcuts.utils.isotime(dtime=None, timespec='seconds', sep=' ')

return ISO format of current timestamp: 2024-04-27 17:31:12

This is shorthand for:

from datetime import datetime
def isotime(dtime=None, timepec='seconds', sep=' '):
    """return ISO format of current timestamp:
          2024-04-27 17:31:12
    """
    if dtime is None:
        dtime = datetime.now()
    return datetime.isoformat(dtime, timespec=timespec, sep=sep)

There isn’t more to it than that, it’s just shorter.

get_homedir(): get home directory

pyshortcuts.utils.get_homedir()

determine home directory

This sort of shorthand for:

pathlib.Path.home().resolve().as_posix()

except that it also checks for SUDO_USER on POSIX systems, and it always uses the win32com module on Windows.

get_cwd(): get current working directory

pyshortcuts.utils.get_cwd()

get current working directory Note: os.getcwd() can fail with permission error.

when that happens, this changes to the users HOME directory and returns that directory so that it always returns an existing and readable directory.

This mostly shorthand for:

pathlib.Path('.').resolve().as_posix()

fix_filename(): turn a string into a valid filename

pyshortcuts.utils.fix_filename(filename, allow_spaces=False)

fix string to be a ‘good’ filename, with very few special characters and (optionally) no more than 1 ‘.’.

More restrictive than most OSes, but avoids hard-to-deal with filenames

argument allow_spaces [default is False] allows spaces in filenames.

Given a string, fix_filename() will return a “good” file name that will work on any operating system. Most of the disallowed or even “inconvenient” characters will be converted to ‘_’. The filename will not have more than 1 ‘.’ character.

new_filename(): make sure a filename is not in use

pyshortcuts.utils.new_filename(filename)

increment filename to be an unused filename

Given a string (perhaps first run through fix_filename()), new_filename() will return a file name that is not in use in the current working folder. Generally, numbers will be incremented in order so that an input of file.001 might become file.002 or file.004 if the interim files exist. If a file named foo.dat exists, the 001 will be inserted before the dot: foo_001.dat. The filenumbers are not limited to 1000.

read_textfile(): read a text file to string

pyshortcuts.utils.read_textfile(filename, size=None)

read text from a file as string

Parameters:

  • file) (filename (str or)

  • None) (size (int or)

Return type:

text of file as string.

Notes

  1. the encoding is detected with charset_normalizer.from_bytes() which is then used to decode bytes read from file.

  2. line endings are normalized to be newlines (’\n’), so that splitting on newline will give a list of lines.

Given a filename or file-like object (io.IOBase instance), this returns a ‘\n’-delimited string from the file. This handles the possibility of different unicode encodings by reading the file contents as bytes, and then using str(charset_normalizer.from_bytest(data).best()) to convert to a string. Line endings of \r and \r\n are replaced by \n.

gformat(): fixed formatting of floating point numbers

pyshortcuts.gformat.gformat(val, length=11)

Format a number with ‘%g’-like format.

Except that:

  1. the length of the output string will be of the requested length.

  2. positive numbers will have a leading blank.

  3. the precision will be as high as possible.

  4. trailing zeros will not be trimmed.

The precision will typically be length-7.

Parameters:

  • val (float) – Value to be formatted.

  • length (int, optional) – Length of output string (default is 11).

Returns:

String of specified length.

Return type:

str

Notes

Positive values will have leading blank.

gformat() converts a floating point number to a string with a specified length, and maximizing the displayed precision for that length. This is very useful for creating tables of floating point numbers.

The formatting will be similar to ‘%g’-like format, expect that:

  1. the length of the output string will be of the requested length.

  2. positive numbers will have a leading blank.

  3. the precision will be as high as possible.

  4. trailing zeros will not be trimmed.

The precision displayed will be determined by the length of the string.

An example:

>>>from pyshortcuts import gformat
>>> gformat(1023/78, length=11)
' 13.1153846'
>>> gformat(10.2, length=11)
' 10.2000000'
>>> gformat(-1/732023, length=11))
'-1.36608e-6'
>>> gformat(-1/732023, length=15)
'-0.000001366077'
>>> gformat(6/80030, length=7)
' 7.5e-5'

sleep(): a higher-precision sleep

Python’s time.sleep function can be somewhat inaccurate, often sleeping 10 or more milliseconds more than requested. Some applications may want a higher-precision precision sleep. The version provided here is just:

def sleep(duration):
    "more accurate sleep()"
    end = perf_counter() + duration
    while perf_counter() < end:
        pass

debugtimer(): debugging runtime of code in a function

Debugging the run time for a function or section of code is a common need, and can be a painful process. Using Python’s timeit module is really good at timing a single statement, but not good at answering “how long is each section of code taking to run”. Sometimes you just want to print out times to find where code is slow. That gets cumbersome to manage.

The debugtimer() helps with this process by creating a DebugTimer object, with a method add() to mark the time with a message, and methods get_report() and show() methods to show a report of total and incremental run times for a section of code. An example usage would be:

import numpy as np
from pyshortcuts import debugtimer, sleep

SHOW_TIMING = True
dtimer = debugtimer('test timer', precision=4)
sleep(0.502)
dtimer.add('slept for 0.500 seconds')
nx = 10_000_000
x = np.arange(nx, dtype='float64')/3.0
dtimer.add(f'created numpy array len={nx}')
s = np.sqrt(x)
dtimer.add('took sqrt')
if SHOW_TIMING:
    dtimer.show()

which would print out a report like:

+----------------------------------+--------------+--------------+
| Message                          |   Delta Time |   Total Time |
+==================================+==============+==============+
| test timer: 2026-05-13 10:30:54  |       0.0000 |       0.0000 |
| slept for 0.500 seconds          |       0.5020 |       0.5020 |
| created numpy array len=10000000 |       0.4010 |       0.9030 |
| took sqrt                        |       0.4349 |       1.3379 |
+----------------------------------+--------------+--------------+

Note that setting SHOW_TIMING to False would suppess the printing of the timing report. This approach can be helpful during development (or even in production code), as the creation of the dtimer object and the dtimer.add() calls add very little runtime cost.

The timing data stored uses Python’s time.perf_counter() function, so should be a high-precision time measurement.

The debugtimer function also stores the number of imported modules (from len(sys.modules))at each event, which can be displayed with the optional use_mod_count option either when createing the debugtimer or when getting the output.

pyshortcuts.debugtimer.debugtimer(title='DebugTimer', with_mod_count=False, verbose=False, precision=3)

debugtimer returns a DebugTimer object to measure the runtime of portions of code, and then write a simple report of the results.

Parameters:

  • title (str, optional title message ['DebugTimer'])

  • precision (int, precision for timing results [4])

  • with_mod_count (bool, whether to include number of imported modules [False])

  • verbose (bool, whether to print() each message when entered [False])

Returns:

  • DebugTimer object, with methods

  • - clear(title=None) (reset Timer)

  • - add(message) (record time, with message)

  • - get_report(tablefmt==’simple_outline’) (return text of report)

  • - show(tablefmt=’simple_outline’) (print timer report)

Example

>>> timer = debugtimer('timer testing', precision=4)
>>> result = foo(x=100)
>>> timer.add('ran function foo')
>>> bar(result)
>>> timer.add('ran bar')
>>> timer.show()

The returned debgugTimer object will have several methods:

class pyshortcuts.debugtimer.DebugTimer(title=None, verbose=False, precision=4, with_mod_count=False)

Measure run times for lines of code and summarize results

Parameters:

  • (str) (title)

  • (bool) (with_mod_count)

  • (int) (precision)

  • (bool)

DebugTimer.clear(title=None)

clear/reset debugtimer, optionally setting the title for the next run.

DebugTimer.add(msg)

add message point to debugtimer

DebugTimer.get_report(precision=None, with_mod_count=None, tablefmt='simple_outline')

get tabular report of results

Parameters:

  • precision (int, precision for timing results [4])

  • with_mod_count (bool, whether to include number of imported modules [False])

  • tablefmt (tabular format style)

Notes

the output is formatted using the tabulate module, and the tablefmt option can be any of the formatting options it supports. An incomplete list of options is:

“plain”, “simple”, “simple_grid”, “pretty”, “psql”, “grid”, “fancy_grid”, “rounded_grid”, “heavy_grid”, “mixed_grid”, “double_grid”, “outline”, “simple_outline”, “rounded_outline”, “heavy_outline”, “mixed_outline”, “double_outline”, “fancy_outline”, “pipe”, “github”, “orgtbl” (Emacs), “rst”,, “mediawiki”, “html”, “textile”, “latex”, “latex_raw”, “latex_booktabs”, “latex_longtable”, “jira”, “tsv”.

DebugTimer.show(precision=None, with_mod_count=False, tablefmt='outline')

print the table generated by get_report