https://doc.rust-lang.org/std/thread/fn.sleep.html
https://doc.rust-lang.org/std/time/struct.Instant.html#method.now
https://rust-lang-nursery.github.io/rust-cookbook/file/read-write.html
https://doc.rust-lang.org/std/fs/struct.File.html#method.open
https://doc.rust-lang.org/std/io/trait.Read.html
https://docs.rs/chrono/latest/chrono/
https://stackoverflow.com/questions/27312069/how-can-i-iterate-over-a-vector-of-functions-and-call-each-of-them
https://doc.rust-lang.org/book/ch12-02-reading-a-file.html
https://stackoverflow.com/questions/26643688/how-do-i-split-a-string-in-rust
https://stackoverflow.com/questions/37888042/remove-single-trailing-newline-from-string-without-cloning
https://stackoverflow.com/questions/14154753/how-do-i-make-an-http-request-from-rust
https://docs.rs/reqwest/0.11.9/reqwest/
https://docs.rs/reqwest/0.11.9/reqwest/blocking/index.html
https://docs.rs/reqwest/0.11.9/reqwest/blocking/struct.Request.html
https://api.slack.com/apps/A03404BKQ1G/incoming-webhooks?success=1
https://doc.rust-lang.org/std/macro.format.html
https://docs.rs/toml/0.5.8/toml/
https://toml.io/en/
https://doc.rust-lang.org/std/net/struct.TcpStream.html
Category: tech
Need these for some part-time work
- Launch Templates https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchTemplates.html
- Spot Fleet Scaling https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-step-scaling.html
- EC2 Auto Scaling Limits https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-capacity-limits.html
- EC2 Spot Instance Requests https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html?icmpid=docs_ec2_console
- Launch template –> AMI User data MIME scripts https://docs.aws.amazon.com/batch/latest/userguide/launch-templates.html#example-mount-an-existing-amazon-efs-file-system
- Static (Elastic) IPs https://aws.amazon.com/premiumsupport/knowledge-center/ec2-associate-static-public-ip/
- AWS EC2 autoscaling (external blog guide) https://www.cloudsavvyit.com/2043/getting-started-with-aws-autoscaling/
- [StackOverflow] literally the point of Spot Instances https://stackoverflow.com/a/11996798/5945794
- AWS SLA https://aws.amazon.com/compute/sla/
- AWS EC2 SSH Connect https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-connect-methods.html
Pricings
- EC2 On Demand https://aws.amazon.com/ec2/pricing/on-demand/
- EC2 Spot Instance pricing history https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances-history.html
- EC2 Spot Pricing current history https://aws.amazon.com/ec2/spot/pricing/
- Reserved EC2 instance https://aws.amazon.com/ec2/pricing/reserved-instances/pricing/
- Cloudwatch https://aws.amazon.com/cloudwatch/pricing/
- EC2 Autoscaling (no charge)
- EBS Storage https://aws.amazon.com/ebs/pricing/
Categories
[aws] intro to automated reasoning
Categories
[hn] docker optimisation guide (top 12)
Categories
[hn] ssh-tools
Categories
new mice [bought]
my mouse is shockingly bad at at higher sensitivity settings so cloud do with a new wirless mouse, currently logitech looking like the likely long term likely choice (anyone know of a synonym for choice that begins with the letter l?)
https://www.logitechg.com/en-gb/products/gaming-mice/g403-hero-gaming-mouse.910-005633.html
update — went for this one becuase (a) it’s cheaper (b) i’m not a cs:go professional so wireless isn’t the end of the world (c) it has a load of utility buttons so i can stop keyboard shortcutting everything.
now just have to hope it runs fine as PnP on linux.
Folks keep telling me I probably have some form of ADHD and the wealth of random pieces of paper that I scribbled on 14 months ago yet still occupy space in my living room may attest to that.
So this might be a worthy £400 expenditure to solve the “paper” problem without buying a full blown tablet.
Categories
atlas – database schema migration cli
i’m sysadmin of some linux gpu servers in the phd working group and having to manually setup new users is a massive PITA.
a few options for replacements
- Auth0 Authentication
- https://github.com/jupyterhub/oauthenticator/blob/master/oauthenticator/auth0.py
- account required
- will it be able to communicate with servers behind firewalled school of computing network?
- KeyCloak OIDC / OAuth2 Authentication
- https://www.keycloak.org/
- functional+tested implementation up and running
- only problem is existing user data — the user’s UID inside the spawned container depends on the order in which users start logging in, rather than ownership of local files.
- could bind mount
/home/{user}into container at/mnt/serverthen change$NB_UIDfor container user duringusermodto match theUIDof the user’s/home/{user}/.bashrcfile — means another change to jupyterhub’sstart.shscript in the Docker images. - Could also symlink
/mnt/serverto/home/{user}/serverat end ofstart.shwith everything else in/home/{user}mounted in a docker volume? although docker volumes are NOT stored on the SSD for some servers because of disk space issue so user data wouldn’t be on the SSD :[ - Better idea would probably be mount
/home/{user}as before and change$NB_UIDof$NB_USERto the uid of.bashrc(withoutchown-ing user files) - If no
/home/{user}exists… then…. what happens? JupyterHub server does create the user account… so should be a simple case of look at uid of.bashrc… see it’s the same as$NB_UID… do nothing…? - BUT … JupyterHub will only create the
/home/{user}directory inside its container (if using the completely containerised version) … which means that/homewould still need to be mounted in thejupyterhub-{type}containers…. This will probably cause an absolute mess for user data though there will be multiple users that have files withUID=1000etc. And because students can still SSH onto the servers it means they might be able to access other user’s data… :[
- Native Authenticator
- https://github.com/jupyterhub/nativeauthenticator
- functional+tested implementation up and running
- Same problems as keycloak, same potential “solution”, i.e. hacky workaround.
https://lists.cd.foundation/g/sig-mlops
This is a public list for the CDF MLOps SIG. All meetings and discussions are held in the open, and everyone is welcome to join. The current membership, calendar, and meeting documents can be found at https://github.com/cdfoundation/sig-mlops.
the Sig-MLOps 2021 roadmap speaks volumes to me and my experience as a machine learning phd student…
At this point in the development of the practice, it perhaps helps to understand that much of ML and AI research and development activity has been driven by Data Science rather than Computer Science teams. This specialisation has enabled great leaps in the ML field but at the same time means that a significant proportion of ML practitioners have never been exposed to the lessons of the past seventy years of managing software assets in commercial environments.
As we shall see, this can result in large conceptual gaps between what is involved in creating a viable proof of concept of a trained ML model on a Data Scientist’s laptop vs what it subsequently takes to be able to safely transition that asset into a commercial product in production environments. It is not therefore unfair to describe the current state of MLOps in 2020 as still on the early path towards maturity and to consider that much of the early challenge for adoption will be one of education and communication rather than purely technical refinements to tooling.
most other phd/professor type folks were not the least bit interested in dealing with these sorts of problems during development — basic codebase documentation often seems to be considered a waste of time based on the hundreds of github repos I’ve looked over.
it’s likely these are a subset of SEP fields and i’ve settled on calling the phenomena “Someone Else Will Clean Up the Mess” fields (or SEWCUM fields for short).
https://github.com/dwmkerr/hacker-laws
the robustness principle is pretty apt for adversarial example research….
https://github.com/dwmkerr/hacker-laws#the-robustness-principle-postels-law